Elon Musk's social media platform X has caused major outrage after individuals utilized its AI chatbot Grok to change pictures of females by stripping their clothing.
Numerous cases have emerged where the technology has been used to digitally undress women to render them in swimwear without permission, as well as to put them in inappropriate scenarios.
Developer Reaction and Regulatory Stance
XAI, the company that created Grok, remained silent to questions, except for an auto-reply response stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to ban nudification software. Under a new law, anyone who provided such tools would "receive a jail term and substantial penalties".
"X or Grok is capable of halting these forms of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the danger" of people accessing unlawful material on their networks, but did not confirm whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium features—which reacts to X users' requests when they mention it in a tweet.
It is frequently used to give commentary or more context to other posters' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The system has been faulted for allowing users to produce images and clips with explicit content and sexually suggestive content, and it was earlier accused of creating graphic material of celebrities.
"The platform has been allowing the production and sharing of these pictures for months without taking any steps and we have not witnessed any action by oversight bodies," the expert added.
XAI's own usage rules bans "depicting images of individuals in a pornographic way".
In a comment, Ofcom said it was against the law to "create or distribute non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "necessary actions" to "minimize the risk" of users encountering illegal material on their services and to delete it quickly when they are notified of it.
