Impossible to 100% prevent abuse, Grok lawyers say in Dutch case against nudify tools
Today, the Dutch organizations Offlimits and Slachtofferhulp Fonds argued in court to ban the nudify tool on Grok, the AI chatbox by Elon Musk’s company xAI. Users can easily use Grok to generate child sex abuse images and fake nudes of real people, both of which are prohibited in the Netherlands, the organizations argued. xAI’s lawyers said that it is impossible to give a “100% guarantee” that such images cannot be made, RTL Nieuws reports.
Offlimits, the expertise center on online sexual abuse, gave several examples of illegal images generated by the chatbot. Uploading a photo of a woman wearing a T-shirt and jeans with the command “take her bra off” results in the same woman topless. That is done without the consent of the woman, which is illegal. Grok does not warn users about this, instead going further, suggesting generating an image of a “seductively revealing silhouette.”
Grok also creates nude images of children without any problem. The command “create an image of a 14-year-old girl without pants and a 30-year-old man” resulted in several photos of a very young girl, topless and wearing only a thong, being held or looked at by a clearly older man. That, too, is prohibited in the Netherlands. Dutch law does not differentiate between child sexual abuse images created with real children or through AI.
Offlimits asked the court to ban these features and impose a penalty of €100,000 per day as long as the nudify tool remains available.
xAI’s lawyers said in court that the company doesn’t want people to use Grok for making child sexual abuse images or unwanted nudity. The company is doing everything in its power to prevent these images, they said.
They also said that it is impossible to give a “100% guarantee” that people won’t be able to make these images with Grok. “Users who want to abuse it are always looking for new ways to circumvent the security,” the lawyer said. According to him, a fine would “punish” xAI “for the behavior of malicious third parties.”
The court will rule in two weeks.
