Mathematicians Demand OpenAI Prove It Didn't Use Their Work

Against the backdrop of a string of impressive mathematical achievements by OpenAI, distrust is mounting in the scientific community. Just days after the first public scandal over the possible use of unpublished work, a second researcher has come forward with grievances against the company, accusing it of unethical and “dishonest” behavior.

What the complaints are about

In a series of posts on Mastodon, mathematician Andreas Thom voiced concern that his own conversations with ChatGPT, as well as inquiries from his colleagues made before the triumphant announcement, may have contributed to the company's success. One of the ten results OpenAI announced last month happened to fall squarely within Thom's field — so-called non-sofic groups (infinite structures that cannot be approximated by finite ones). At first OpenAI made no mention of the recent contributions of Thom and Gábor Kun, for which it drew criticism and later quietly amended the text, acknowledging that the result relied heavily on their work.

What put Thom on guard was the “detailed command of our techniques,” which, by his account, were at the time neither the most obvious nor the most promising paths to a solution. He wrote to OpenAI employees Sébastien Bubeck and Mark Sellke, asking whether his conversations with the chatbot could have ended up in the training data or been accessible during the model's reasoning process.

An answer that fell short

The reply Thom received did not satisfy him: it addressed only direct access to the correspondence, not whether it had made its way into the general body of training data. “No qualifications, explanations, or proof were offered. I regard this as dishonesty, to put it mildly,” he wrote. Researchers, Thom stresses, are in no position to reconstruct OpenAI's training pipeline — only the company itself holds the relevant information, and so the burden of proof falls precisely on it.

The same evasive wording, Thom notes, was heard in connection with OpenAI's recent breakthrough on one of the “millennium problems” — the Navier–Stokes equations. The company flatly denied using specific user data, but added that, while unlikely, it could not be ruled out that “de-identified data” from product use had helped improve the models. “De-identification removes the name, but it does not remove the intellectual content of a mathematical idea,” Thom shot back.

A bad aftertaste in the community

According to Thom, it would be “ethically unacceptable” if users' non-public research improved the very models the company then used to outpace those same users in the race to publish — without consent, disclosure, or credit. OpenAI did not immediately respond to The Verge's request for comment.

The episode has left a bad aftertaste at a moment that was meant to be a triumphant one for the company. Many mathematicians fear that such behavior will nudge the field toward greater secrecy: if even rumors of an imminent breakthrough are enough to trigger a race with a wealthy tech giant hungry for glory, people will become far more cautious about sharing ideas.

Source: The Verge