xAI Accused of Training Grok on Child Sexual Abuse Material

A lawsuit has been filed against xAI, Elon Musk's company, alleging that the Grok neural network was trained on child sexual abuse material (CSAM). It is the first case of its kind against xAI, and, as Ars Technica notes, the complaint is sparing on the specifics of the training allegation itself.

The plaintiff, proceeding under the pseudonym Jane Doe, recounts that in the early 2000s, when she was a preschool-age child, she was victimized: adult men raped her in order to produce and sell CSAM online. Since then, her images have been entered into the hash lists of organizations such as the National Center for Missing & Exploited Children (NCMEC) and the Canadian Centre for Child Protection (CCCP).

Doe is subscribed to alerts from the U.S. Department of Justice's victim notification system. She says she was shaken when the CCCP informed her that AI-generated CSAM images featuring her had been found on the xAI platform. According to the complaint, offenders on forums discussed creating such images using the likenesses of “well-known, long-standing” victims.

How the claim is structured

The plaintiff's central concern is that xAI, she alleges, stores the images Grok generates and uses them to further train the model. Under the service's terms, public posts on X and Grok's own outputs are by default treated as training data. Publishing an image, then, does not merely make it viewable but feeds it directly into the training pipeline.

The lawsuit notes that xAI filters scenes of violence out of its training data, yet the terms of service do not state whether CSAM, non-consensual intimate imagery (NCII), or NSFW material is excluded. Hence the complaint's conclusion: it is technically difficult to completely remove the influence of a training example from an already-trained model, and any material that made it into training likely continues to shape outputs even after the source images are deleted.

As for training on the original real photographs, the wording is more cautious here: the attorneys assert that Doe's images were on the NCMEC hash list and that “the same material” was allegedly part of the dataset on which xAI built Grok's image- and video-generation capabilities.

The demands

Doe filed a putative class action on behalf of all victims whose childhood images may have been used to generate CSAM in Grok. X is accused of violating federal child pornography laws and the so-called “Masha's Law,” which give survivors the right to sue for the production, possession, and distribution of such material. In the words of attorney Margaret Mabie, “xAI has done all three.”

The plaintiff demands the destruction of all material stored on the servers, an order barring Grok from generating CSAM — which, in her view, would require blocking any sexualized outputs, including NCII and NSFW images — as well as compensation for those harmed. X did not respond to the publication's request for comment.

Source: Ars Technica