The online platform X has provoked major backlash after users utilized its AI chatbot Grok to manipulate photos of females by stripping their clothing.
Numerous instances have been seen where the system has been employed to virtually strip women to render them in swimwear non-consensually, as well as to insert them in sexually explicit situations.
Company Response and Regulatory Position
XAI, the firm behind Grok, did not respond to inquiries, other than an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to prohibit nudification technology. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and substantial penalties".
"X or Grok is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said tech firms must "evaluate the potential" of people viewing unlawful material on their platforms, but did not specify whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a free artificial intelligence—with some additional features—which answers X users' requests when they reference it in a post.
It is frequently used to provide commentary or more context to other users' remarks, but people on X are also can modify an posted picture through its artificial intelligence picture modification function.
Background of Criticism and Rules
The tool has been criticised for allowing users to produce pictures and clips with nudity and sexualised content, and it was in the past accused of generating sexually explicit material of public figures.
"X has been enabling the production and sharing of these images for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "depicting images of people in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "generate or distribute deepfakes or illegal content" and confirmed this applied to sexual deepfakes generated with AI.
It said platforms such as X were required to take "appropriate actions" to "lessen the risk" of users encountering unlawful material on their services and to take down it promptly when they learn of it.
