The online network X has provoked significant anger after people employed its AI chatbot Grok to manipulate pictures of females by taking off their attire.
Several cases have been seen where the tool has been employed to digitally undress women to show them in swimwear without permission, as well as to place them in inappropriate contexts.
Company Response and Legal Position
XAI, the company behind Grok, remained silent to inquiries, aside from an auto-reply response stating "traditional news outlets lies".
A government representative stated that new laws is being brought forward to prohibit nudification software. Under a new criminal offence, anyone who supplied such technology would "receive a jail term and substantial penalties".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their platforms, but did not confirm whether it was at this time looking into X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional functions—which responds to X users' requests when they reference it in a tweet.
It is frequently used to give commentary or more context to other posters' remarks, but people on X are also can modify an posted image through its AI image editing function.
Background of Controversy and Rules
The chatbot has been condemned for permitting users to create pictures and videos with nudity and sexualised content, and it was earlier accused of generating graphic material of public figures.
"X has been enabling the creation and distribution of these pictures for a long time without taking any action and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and confirmed this covered explicit AI images made with AI.
It said platforms such as X were required to take "suitable actions" to "lessen the danger" of users viewing unlawful material on their platforms and to take down it promptly when they learn of it.
