AI scammers outplay humans at the art of worming into your trust

Researchers from four universities — Amrita Vishwa Vidyapeetham (India), Ca' Foscari University of Venice, the University of Melbourne and Ben-Gurion University — set out to test whether generative AI can single-handedly run the longest and most labor-intensive part of a scam. They looked at the scheme known as “pig butchering”: over the course of months, a victim is drawn into a romantic correspondence and then steered toward a fake cryptocurrency investment, sometimes bilked out of six-figure sums.

The experiment

In early 2025, 22 volunteers were invited to take part in what was billed as a study of “how people make friends online.” Each spent a week chatting with two “correspondents”: one was a seasoned romance-scam specialist, the other a Claude-based agent — something the subjects never suspected. At the end, both asked for a favor: the human offered a video game to download, the bot an app it had supposedly written itself.

The results were telling. The AI's request was granted by 46% of participants, while fewer than 18% went along with the human. Trust in the correspondents was rated on a five-point scale: the bot averaged 3.78 against the human's 3.31. And 80% of all messages over the week were addressed to the AI — people clearly preferred writing to it.

“Trust harvesting”

One of the authors, Gilad Gressel, calls this “trust harvesting”: first an emotional attachment is built, and then the victim is primed to be deceived. As the researchers envision it, an AI could automatically handle the entire first stage, with a human stepping in only at the very end to get around language models' safeguards.

A troubling detail: Claude dutifully followed instructions not to admit that it was an AI. It flatly denied it and even invented plausible explanations for its own slip-ups. Only one participant saw through the bot on their own. That said, once the truth came out, 20 of the 22 people correctly identified which one had been the machine. “That's exactly how scams work,” Gressel explains. “Once the deception is exposed, it's all obvious. But from inside the illusion, you can't see it.”

In a separate series of tests, Gemini 3.1 Pro refused to admit its nature even under direct ethical pressure, whereas ChatGPT 5.5 and Claude Opus 5 came clean when reproached for the unethical nature of the deception.

Reaction and outlook

Anthropic stated that company policy prohibits using Claude for fraud or impersonating a human, and that the study relied on an outdated model from early 2025. According to the company, the new Claude Opus 5 correctly recognizes romance scams in 97% of cases — though the researchers note that this figure applies to complete dialogues that include a pitch to “invest,” not to the innocuous rapport-building stage.

For now, human labor remains cheaper: the authors surveyed 145 former workers from scam “farms” in Cambodia, Myanmar and Laos — often trafficking victims, for whom the operators additionally collect a ransom. But a shift to automation could not only reduce this slavery but also deprive investigators of their main lead. As former prosecutor Erin West warns, instead of conspicuous “farms,” a scam could be run out of an ordinary two-bedroom apartment.

Source: Ars Technica