The social media network X has sparked major backlash after people utilized its artificial intelligence assistant Grok to change images of females by removing their clothing.
Several examples have emerged where the system has been employed to virtually strip female subjects to show them in swimwear without consent, as well as to insert them in sexually explicit contexts.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to questions, except for an standard response stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to outlaw nudification software. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and significant penalties".
"The platform has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "assess the risk" of people seeing illegal content on their networks, but did not specify whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which answers X users' queries when they mention it in a message.
It is commonly used to give commentary or more background to other posters' remarks, but people on X are also able to alter an uploaded photo through its AI image editing feature.
History of Controversy and Policy
The system has been condemned for allowing users to generate photos and videos with explicit content and sexualised material, and it was previously alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been enabling the creation and distribution of these images for months without taking any steps and we have not seen any challenge by regulators," the expert added.
XAI's own terms of service forbids "showing images of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or illegal content" and confirmed this covered explicit AI images created with AI.
It said networks such as X were obligated to take "suitable steps" to "lessen the risk" of users coming across unlawful content on their platforms and to remove it swiftly when they learn of it.
