OpenAI Shut Down a Cambodian Fraud Network That Was Using ChatGPT

OpenAI has announced that it blocked the activities of an organized group of social engineers from Cambodia that had been using ChatGPT for a whole range of fraudulent schemes. Security expert Bruce Schneier wrote about the scale of the operation on his blog.

An assembly line of deception

What set the network apart was that it ran several types of scam at once, often blending elements of different schemes. The operators used the personas of dating-site members to win a victim's trust, then invited them to take part in fraudulent investments — in cryptocurrencies or in spot-market gold trading.

Other members of the network kept up drawn-out romantic correspondence in the guise of invented characters, passed themselves off as representatives of online casinos promising bogus bonuses and winnings, or posed as law enforcement officers telling victims they had to pay a fine for grave crimes they had supposedly committed.

One hand behind every role

The scenarios varied, but behind them lay one and the same pattern of behavior. The scammers created fake profiles on dating sites and dreamed up personas of fictitious investment experts and phony law enforcement officers. With the help of the neural network they also generated images of forged documents: passports, legal notices, confirmations of stock purchases, and gambling-platform interfaces.

This story is a good illustration of a point that is heard more and more often around large language models. One of the commenters on Schneier's blog wryly observed that LLMs have finally found their calling: their true strength lies not in producing accurate and trustworthy information — they still struggle with that — but in plausibly bamboozling gullible people and stripping them of their savings. The ability to convincingly invent things that don't exist turns out to be the perfect tool for a fraudster.

Source: Schneier on Security