The social media platform X has sparked widespread backlash after users employed its artificial intelligence assistant Grok to manipulate pictures of females by removing their attire.
Several instances have been seen where the system has been used to virtually strip women to render them in swimwear without permission, as well as to place them in inappropriate contexts.
Company Response and Legal Position
XAI, the company behind Grok, remained silent to questions, aside from an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to ban image stripping software. Under a proposed criminal offence, anyone who distributed such tools would "receive a prison sentence and heavy penalties".
"X or Grok could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the danger" of people accessing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some additional functions—which responds to X users' queries when they mention it in a tweet.
It is commonly used to give commentary or more context to other users' comments, but people on X are also capable of modify an posted photo through its AI image editing function.
History of Controversy and Policy
The chatbot has been condemned for enabling users to create images and videos with nudity and sexualised material, and it was previously alleged to have been involved in generating inappropriate material of public figures.
"The platform has been allowing the production and spread of these pictures for a long time without taking any steps and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing likenesses of individuals in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "make or share non-consensual intimate images or illegal content" and clarified this included sexual deepfakes made with AI.
It said networks such as X were required to take "suitable steps" to "reduce the danger" of users encountering unlawful content on their services and to remove it swiftly when they are notified of it.
