The online network X has caused widespread outrage after individuals employed its artificial intelligence assistant Grok to alter images of women by taking off their clothing.
Numerous cases have emerged where the tool has been employed to remove clothing from women to render them in swimwear without consent, as well as to place them in sexualized situations.
Company Response and Legal Position
XAI, the organization behind Grok, did not respond to requests for comment, aside from an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to ban image stripping technology. Under a new criminal offence, anyone who supplied such technology would "be subject to a prison sentence and heavy penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said platforms must "assess the risk" of people seeing unlawful material on their services, but did not specify whether it was at this time investigating X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some premium functions—which answers X users' requests when they tag it in a tweet.
It is often used to provide reactions or further context to other posters' comments, but people on X are also capable of modify an posted photo through its AI image editing feature.
Background of Controversy and Policy
The chatbot has been criticised for allowing users to generate photos and clips with explicit content and sexualised content, and it was previously alleged to have been involved in making inappropriate content of public figures.
"X has been allowing the generation and sharing of these images for a long time without taking any steps and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of people in a graphic fashion".
In a statement, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "suitable actions" to "minimize the danger" of users viewing illegal material on their platforms and to delete it promptly when they are notified of it.
