“I Didn't Know AI Could Make Up Facts”: Lawyer Punished for Fake Testimony from ChatGPT

The New Mexico Supreme Court found attorney Steven Aarons guilty of direct contempt of court: in an appellate brief he cited “false testimony from wholly fabricated witnesses.” The “witnesses” included, among others, officers Michelle Amarillo and Sanchez, as well as Manal Al-Jibouri and Theresa Marquez — people who were never at the trial. Some of the real testimony the attorney attributed to the wrong individuals, and he described the precedents in the brief incorrectly.

Aarons, who had defended clients over more than 40 years of practice, took on an appeal in a murder case. He fed ChatGPT (by his account, the version running on OpenAI's o3 model) a computer transcript of the trial and other materials, then signed the resulting text and filed it with the court without checking either the facts or the citations to the law. He reported the problem neither to his client nor to the court — until prosecutors themselves moved to strike parts of the brief.

“You Buried Your Head in the Sand”

The justices were merciless. Justice Shannon Bacon was especially harsh, counting “at least eight or nine” violations of the code of professional ethics. Aarons's admission that he had not known AI was capable of making up facts did not convince her: everyone knows about neural-network “hallucinations,” she noted — including her 13-year-old nephew and her 75-year-old stepmother. “Do you watch the news at all, listen to the radio, read anything? Stories about lawyers relying on AI's fabrications are front-page news every day,” Bacon said. As she saw it, the attorney had two options — to remain willfully ignorant or to take the risk, and both are incompatible with the code.

The court stressed that there is no difference between an invented precedent and a real one recounted incorrectly — the duty of truthfulness applies equally. Nor does it matter who prepared the text: an AI, a first-year student, or a colleague. A signature on a brief means the lawyer vouches for its accuracy.

What troubled the justices most was the fate of the defendant. “It's your client who suffers from this far more than you ever will,” Bacon said, noting that the convicted man remains in custody until the case is resolved. Aarons, moreover, had not disclosed the substance of the problem to his client, telling his relatives only that “there were difficulties with the first brief,” while saying nothing about ChatGPT.

The Outcome

The court fined the attorney $5,000, payable to the client protection fund, barred him from appearing before the state Supreme Court until the proceedings conclude, and referred the matter to the disciplinary commission, where further sanctions are possible. All of the earlier briefs were struck from the record, and the defendant's representation was assigned to a public defender. Asked whether Aarons would be paid for his work, Justice Bacon replied curtly: “I'm not sure that's going to happen.”

Source: Ars Technica