68% of Peer-Reviewed Papers Turned Out to Be AI Slop: A View from the Trenches

Caleb Robinson and Isaac Corley, specialists in geospatial machine learning, reviewed 22 papers this summer that had been submitted to NeurIPS, WACV and the TerraBytes workshop at ECCV. They describe the experience as life “in the trenches of slop”: 15 of the 22 papers (68%) contained entirely fabricated citations, doctored author lists on genuinely existing papers, or were obviously generated by a language model — with hallucinated jargon and incoherent text.

Impostor authors made it through review

The most telling case: in two papers the authors of real articles were swapped out for nonexistent researchers — with names close enough to survive a cursory glance. For instance, the authors of the SatMAE paper, “Yezhen Cong, Samar Khanna, Chenlin Meng,” were turned into “Yuyang Cong, Saurabh Khanna, Chen Meng.” The reviewer recommended rejecting both papers and reported the forgery to the organizers. Both were accepted for oral presentations — on the condition that they simply fix the citations.

There were other finds too: a 53-page NeurIPS submission stuffed with made-up jargon, and a paper in which the model's own working notes had been left in alongside the references.

The scale of the problem

The authors cite outside estimates. An analysis by Nature found tens of thousands of 2025 publications with unreliable AI citations; an audit of preprint servers counted around 146,900 hallucinated references over the course of a year, and 85.3% of them survived into the published versions as well — peer review does not weed them out. An audit by The Lancet of 2.5 million biomedical articles showed a sixfold increase in the share of papers with at least one fake citation: from one in 2,828 papers in 2023 to one in 277 in early 2026.

The flow runs the other way as well. According to Pangram, 21% of the reviews for ICLR 2026 were entirely AI-generated. At ICML 2026, hidden prompt-injection “traps” were embedded in the papers' text, catching 795 reviews from authors who had been barred from using models. A separate study showed that adversarially reworking an abstract without changing its scientific content raised the score from AI reviewers in roughly 38% of cases.

What the authors propose

Robinson and Corley stress that they themselves use language models every day, but they proofread and rewrite the output by hand. Their complaint is not with the tool but with papers the authors themselves put no time into. They consider disclosing the use of AI a half-measure: the conscientious will admit it, the unscrupulous won't. What's needed is a mechanism for automatic rejection before review even begins.

As a first step, they released bib-audit — an open-source (MIT) skill for Claude Code that checks every citation against Crossref, arXiv, DataCite and Semantic Scholar, spotting nonexistent works, fabricated identifiers and swapped-out authors. They recommend running it on your own draft before submission — so that it doesn't have to be run for you during review.

Source: Hacker News