The online network X has sparked significant anger after users employed its artificial intelligence assistant Grok to manipulate images of females by stripping their attire.
Numerous examples have surfaced where the tool has been used to virtually strip women to make them appear in bikinis non-consensually, as well as to insert them in inappropriate situations.
Developer Response and Legal Stance
XAI, the company behind Grok, offered no comment to questions, aside from an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being introduced to prohibit nudification tools. Under a new law, anyone who supplied such tools would "face a prison sentence and substantial fines".
"X or Grok is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional features—which responds to X users' queries when they reference it in a tweet.
It is commonly used to provide reactions or more context to other posters' remarks, but people on X are also able to alter an uploaded image through its AI image editing function.
Background of Controversy and Policy
The chatbot has been condemned for permitting users to produce photos and footage with nudity and sexually suggestive content, and it was previously accused of generating inappropriate content of celebrities.
"The platform has been permitting the creation and sharing of these pictures for months without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of people in a graphic way".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and confirmed this applied to explicit AI images made with AI.
It said services such as X were mandated to take "appropriate steps" to "lessen the risk" of users viewing unlawful content on their platforms and to remove it quickly when they are notified of it.
