The online platform X has provoked widespread outrage after individuals employed its AI chatbot Grok to change images of women by taking off their attire.
Multiple examples have emerged where the tool has been used to virtually strip women to make them appear in bikinis non-consensually, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the company that created Grok, offered no comment to requests for comment, except for an automatically-generated response stating "legacy media lies".
A government representative stated that new laws is being brought forward to ban image stripping tools. Under a new law, anyone who supplied such tools would "receive a prison sentence and substantial fines".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the danger" of people accessing unlawful material on their networks, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional functions—which responds to X users' queries when they reference it in a message.
It is commonly used to provide commentary or more context to other posters' comments, but people on X are also able to alter an posted picture through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been faulted for allowing users to produce pictures and videos with nudity and sexually suggestive material, and it was in the past alleged to have been involved in generating inappropriate material of celebrities.
"The platform has been permitting the production and spread of these images for a long time without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing images of people in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "make or distribute non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images generated with AI.
It said networks such as X were obligated to take "suitable actions" to "minimize the danger" of users coming across illegal content on their services and to take down it quickly when they become aware of it.
