The social media network X has provoked major anger after users used its AI chatbot Grok to alter images of females by stripping their attire.
Multiple instances have been seen where the tool has been used to virtually strip female subjects to show them in swimwear without permission, as well as to place them in inappropriate contexts.
Developer Response and Legal Stance
XAI, the company that created Grok, offered no comment to inquiries, aside from an standard message stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to ban image stripping software. Under a proposed criminal offence, anyone who supplied such tools would "face a prison sentence and substantial penalties".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the potential" of people seeing illegal content on their services, but did not confirm whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some additional features—which reacts to X users' queries when they reference it in a tweet.
It is often used to offer commentary or more context to other posters' comments, but people on X are also capable of edit an posted photo through its AI image editing function.
History of Criticism and Rules
The chatbot has been condemned for allowing users to create pictures and clips with explicit content and sexualised material, and it was in the past accused of creating graphic material of public figures.
"X has been allowing the creation and spread of these images for a long time without taking any measures and we have not witnessed any action by regulators," the expert added.
XAI's own terms of service forbids "portraying likenesses of persons in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "make or share non-consensual intimate images or child sexual abuse material" and clarified this applied to explicit AI images made with AI.
It said services such as X were obligated to take "necessary actions" to "minimize the danger" of users viewing illegal material on their services and to take down it quickly when they become aware of it.
