Elon Musk's online network X has provoked widespread anger after people utilized its artificial intelligence assistant Grok to change photos of females by taking off their garments.
Multiple examples have been seen where the tool has been employed to remove clothing from female subjects to render them in swimwear non-consensually, as well as to put them in sexualized scenarios.
Company Response and Regulatory Stance
XAI, the company behind Grok, did not respond to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban nudification tools. Under a proposed law, anyone who distributed such technology would "face a prison sentence and substantial penalties".
"The platform has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "assess the risk" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some additional features—which responds to X users' prompts when they tag it in a tweet.
It is commonly used to offer commentary or more context to other users' comments, but people on X are also capable of edit an uploaded picture through its AI image editing function.
History of Criticism and Policy
The system has been faulted for permitting users to produce images and videos with explicit content and sexually suggestive material, and it was previously alleged to have been involved in creating sexually explicit content of celebrities.
"X has been permitting the creation and distribution of these images for a long time without taking any steps and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing images of people in a graphic way".
In a statement, Ofcom said it was against the law to "create or distribute private pictures or child sexual abuse material" and stated this included explicit AI images created with AI.
It said services such as X were obligated to take "appropriate actions" to "minimize the risk" of users viewing illegal content on their services and to delete it promptly when they are notified of it.
