Dutch Court Orders X, Grok to Stop AI-generated Sexual Abuse Content

A Dutch court has ordered X and its AI chatbot Grok to immediately stop generating non-consensual sexualized imagery and child pornographic material in the Netherlands, imposing a penalty of €100,000 per day on each defendant for non-compliance.

The judgment, issued Thursday by the Amsterdam District Court, is the first European court ruling to impose a binding injunction on an AI image generator over non-consensual sexualized content.

The case was brought by Offlimits, an expertise center on online sexual abuse, and Fonds Slachtofferhulp, a victim support organization, after Offlimits concluded that regulatory enforcement was moving too slowly relative to the pace of harm. Dutch law already prohibits the creation and distribution of deepfake nude images without consent. After Grok's image editing feature launched in late December 2025 and reports to Offlimits surged, the organization sent a formal notice to xAI and X on February 4, 2026, demanding they cease the functionality.

The court ordered xAI (operating as Grok) to stop generating and distributing sexual imagery of persons residing in the Netherlands without their explicit consent, and to stop producing, distributing, or publicly displaying content that qualifies as child pornographic material under Dutch law. X Corp and X Internet Unlimited Company (XIUC) — X's EU-facing entity — are separately ordered to stop offering Grok's functionality as part of the X platform for as long as those violations persist.

“Grok and X are ordered to cease offering the undressing functionality that makes it possible to undress Dutch people. This means that Grok and X have to stop offering this functionality not only in the Netherlands. But everywhere,” advocate Karlijn Han, representing Offlimits, told Tech Policy Press.

xAI must also confirm in writing to Offlimits how it has complied. Failure to do so triggers the same daily penalty.

What the court found

The ruling hinges on a narrow but decisive question of whether the defendants had made the generation of this content impossible, as they claimed, or whether reasonable doubt remained. The court found they had not.

At the March 12 hearing, xAI's lawyers argued the company had implemented stringent safeguards as of January 20, 2026, categorically rejecting any suggestion that Grok still permitted non-consensual intimate imagery or child sexual abuse material.

The court found that claim difficult to reconcile with evidence submitted by Offlimits showing that on March 9, 2026 — the same day the defendants sent that categorical denial — Offlimits was still able to generate a video of an existing person in a sexualized context, from a single uploaded photograph, without Grok verifying consent.

"The fact that generating this video was apparently still possible on the same day that the defendants wrote to Offlimits categorically rejecting any suggestion that such content can be generated raises reasonable doubt regarding the certainty with which the defendants stated that the measures taken are adequate," the judgment reads.

On the child sexual abuse material claims, the court also noted a structural inconsistency in xAI's position. The company simultaneously argued that generating such material was now impossible and that a 100% guarantee was technically unachievable.

On the legal basis for the injunctions, the court held that non-consensual undressing images constitute a violation of the GDPR, and that the facilitation of child pornographic material constitutes unlawful conduct under Article 6:162 of the Dutch Civil Code. Critically, the court rejected xAI's argument that liability lay with the users who issue the prompts rather than with the platform. As an internet intermediary with control over Grok's functionalities, the court held, xAI is the appropriate party to prevent the generation and distribution of unlawful images — regardless of whether it is independently liable alongside users.

The ruling lands squarely within the framework established by the Court of Justice of the European Union's Russmedia judgment, handed down on December 2, 2025. In that Grand Chamber ruling, the ECJ held that online platform operators are joint controllers under the GDPR for user-generated content involving sensitive personal data, and cannot invoke DSA liability exemptions to escape GDPR obligations. The Amsterdam court applied the same logic, finding that XIUC and xAI qualify as controllers for the processing of personal data involved in the undressing functionality, and that Dutch courts have jurisdiction under Article 79(2) GDPR.

The Grok case, however, goes further. Russmedia involved a platform that passively hosted harmful content submitted by an anonymous user. Grok generates the content itself. The court addressed this directly, ruling that xAI's role as the designer and operator of the image generation model makes it the "designated party" to prevent unlawful outputs, regardless of who issues the prompt.

The court drew one significant limitation. Because X Corp is a US entity that does not directly offer services in the Netherlands, its injunction is narrower than XIUC's: X Corp is only prohibited from offering Grok's functionality on platform X while Grok violates the non-consensual undressing ban — not the child sexual abuse material ban, since the Dutch court found it lacked international jurisdiction over harms involving generated images of fictitious persons outside the Netherlands. XIUC, as X's EU-facing entity, faces both prohibitions in full.

Grok continues to face regulatory scrutiny

The case was brought following a report by the Center for Countering Digital Hate estimating that Grok generated approximately 3 million sexualized images between December 29, 2025, and January 8, 2026, including an estimated 23,000 appearing to depict children.

Thursday's ruling arrives alongside parallel enforcement actions from Ireland's Data Protection Commission, which opened a GDPR investigation in February; the European Commission, which launched DSA proceedings against X in January; and Ofcom, which opened an Online Safety Act investigation in the UK. Civil suits have also been filed in the United States this month, including by three Tennessee teenagers on March 16 and the city of Baltimore on Wednesday.

Earlier on Thursday, the European Parliament voted to adopt its position on amending the AI Act to include an explicit ban on nudifier systems.

The defendants have ten working days from service of the judgment to confirm in writing to Offlimits how they have complied.

Tech Policy Press reached out to X for comment, but did not receive a reply before publication.