Elon Musk's online network X has caused major backlash after users used its AI chatbot Grok to alter photos of women by removing their clothing.
Numerous cases have emerged where the technology has been used to digitally undress women to render them in swimwear non-consensually, as well as to insert them in sexualized situations.
Company Response and Legal Position
XAI, the organization that created Grok, did not respond to inquiries, other than an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to ban nudification tools. Under a new law, anyone who provided such tools would "receive a jail term and substantial penalties".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for features—which reacts to X users' requests when they tag it in a post.
It is often used to give reactions or further context to other posters' comments, but people on X are also capable of edit an uploaded picture through its artificial intelligence picture modification function.
History of Criticism and Policy
The system has been criticised for permitting users to produce images and footage with explicit content and sexualised content, and it was earlier accused of generating inappropriate material of celebrities.
"The platform has been permitting the creation and sharing of these images for months without taking any action and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting likenesses of individuals in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "make or circulate non-consensual intimate images or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.
It said services such as X were mandated to take "suitable steps" to "minimize the risk" of users viewing illegal content on their services and to remove it promptly when they are notified of it.
