The social media network X has sparked widespread anger after individuals used its AI chatbot Grok to alter photos of females by taking off their attire.
Several instances have emerged where the tool has been employed to virtually strip women to make them appear in swimwear without consent, as well as to insert them in sexually explicit situations.
Company Response and Regulatory Position
XAI, the firm behind Grok, did not respond to inquiries, aside from an auto-reply response stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to outlaw nudification tools. Under a new criminal offence, anyone who distributed such technology would "receive a prison sentence and heavy fines".
"The platform has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the potential" of people accessing illegal content on their platforms, but did not specify whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some paid-for functions—which reacts to X users' prompts when they mention it in a tweet.
It is frequently used to provide commentary or additional background to other users' comments, but people on X are also capable of edit an uploaded image through its AI image editing feature.
Background of Criticism and Rules
The system has been condemned for permitting users to produce pictures and videos with explicit content and sexualised content, and it was in the past alleged to have been involved in generating graphic content of public figures.
"The platform has been allowing the production and sharing of these pictures for a long time without taking any action and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting images of persons in a graphic way".
In a statement, Ofcom said it was illegal to "create or circulate private pictures or child sexual abuse material" and clarified this applied to explicit AI images generated with AI.
It said networks such as X were required to take "necessary steps" to "minimize the risk" of users coming across unlawful content on their services and to remove it swiftly when they learn of it.
