The online platform X has sparked widespread anger after users used its AI chatbot Grok to change pictures of females by removing their garments.
Multiple instances have been seen where the technology has been employed to virtually strip women to show them in swimwear non-consensually, as well as to insert them in inappropriate situations.
Company Reaction and Legal Position
XAI, the organization responsible for Grok, did not respond to questions, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to prohibit nudification software. Under a new criminal offence, anyone who supplied such technology would "be subject to a prison sentence and substantial fines".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people viewing unlawful material on their services, but did not confirm whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional functions—which responds to X users' queries when they tag it in a message.
It is commonly used to offer reactions or further context to other posters' comments, but people on X are also can alter an uploaded photo through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The tool has been criticised for allowing users to generate pictures and videos with explicit content and sexualised content, and it was earlier accused of generating inappropriate material of public figures.
"X has been permitting the generation and spread of these pictures for months without taking any steps and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a graphic manner".
In a statement, Ofcom said it was against the law to "create or distribute non-consensual intimate images or illegal content" and stated this included explicit AI images generated with AI.
It said platforms such as X were obligated to take "appropriate actions" to "lessen the danger" of users encountering illegal material on their platforms and to remove it quickly when they become aware of it.
