Elon Musk’s Grok blocked in NL from creating AI-generated nude images of children

A Dutch court has prohibited the chatbot Grok from offering its “strip function” in the Netherlands. The ban was imposed on Thursday following a lawsuit filed by the Victim Support Fund and the Offlimits expertise center.

Grok was developed by xAI, the American company owned by businessman Elon Musk. The chatbot allowed users to generate AI-driven nude imagery, ranging from depictions of adults to children. These subjects may be entirely virtual, existing only online, or they may be 'deepfakes’, fabricated images of real people.

To strengthen their case, the Victim Support Fund and Offlimits presented images they generated using Grok, depicting underdressed virtual children alongside adult men.

The judge stated that the makers of Grok “probably and negligently contribute to a climate of online abusive behavior.” Since xAI controls Grok, it is responsible for preventing misuse, even if the platform’s creators are not directly generating the images.

The prohibition covers images of individuals residing in the Netherlands, as well as their creation and distribution within the country. While the organizations sought a global ban on the “nudification tool,” the court declined to impose it.

During the hearing, Grok’s attorneys argued that the company aims to prevent child exploitation but that completely stopping all misuse is impossible. “There will always be users who circumvent security safeguards,” they said.