Elon Musk's online network X has provoked significant outrage after people utilized its AI chatbot Grok to manipulate pictures of females by stripping their garments.
Multiple cases have surfaced where the tool has been used to remove clothing from women to show them in bikinis without consent, as well as to put them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the company behind Grok, offered no comment to requests for comment, except for an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being proposed to ban nudification technology. Under a new law, anyone who distributed such tools would "face a jail term and heavy penalties".
"The platform is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people seeing illegal content on their platforms, but did not confirm whether it was at this time investigating X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some additional functions—which reacts to X users' queries when they mention it in a message.
It is frequently used to give commentary or additional background to other posters' remarks, but people on X are also capable of alter an uploaded photo through its AI image editing tool.
History of Criticism and Policy
The tool has been faulted for enabling users to generate pictures and footage with explicit content and sexualised material, and it was in the past alleged to have been involved in creating graphic material of celebrities.
"The platform has been permitting the creation and sharing of these pictures for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "portraying images of people in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images made with AI.
It said networks such as X were required to take "necessary measures" to "lessen the danger" of users coming across illegal content on their platforms and to take down it promptly when they are notified of it.
