The social media platform X has sparked major anger after people used its AI chatbot Grok to change photos of women by stripping their clothing.
Several examples have emerged where the tool has been employed to virtually strip female subjects to render them in swimwear without permission, as well as to insert them in sexually explicit situations.
Company Response and Legal Position
XAI, the company that created Grok, remained silent to requests for comment, except for an auto-reply response stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to prohibit nudification tools. Under a new criminal offence, anyone who distributed such technology would "face a prison sentence and substantial fines".
"X or Grok could prevent these types of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people seeing illegal content on their services, but stopped short of confirming whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some paid-for functions—which reacts to X users' requests when they mention it in a message.
It is commonly used to offer commentary or additional background to other posters' remarks, but people on X are also able to modify an uploaded picture through its AI image editing feature.
History of Controversy and Policy
The tool has been faulted for enabling users to produce images and footage with explicit content and sexualised content, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"X has been enabling the production and sharing of these pictures for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of individuals in a graphic fashion".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said platforms such as X were obligated to take "necessary actions" to "minimize the danger" of users viewing unlawful content on their services and to remove it promptly when they are notified of it.
