Elon Musk's online platform X has sparked widespread backlash after people employed its AI chatbot Grok to manipulate photos of females by removing their attire.
Several cases have been seen where the system has been employed to digitally undress female subjects to show them in swimwear non-consensually, as well as to insert them in sexually explicit scenarios.
Company Reaction and Regulatory Stance
XAI, the company that created Grok, remained silent to questions, other than an automatically-generated message stating "legacy media lies".
A Home Office official stated that legislation is being proposed to ban nudification software. Under a new criminal offence, anyone who supplied such tools would "be subject to a jail term and substantial penalties".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was presently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional features—which answers X users' requests when they tag it in a post.
It is often used to provide reactions or further context to other posters' comments, but people on X are also able to modify an uploaded picture through its AI image editing tool.
History of Criticism and Rules
The chatbot has been faulted for enabling users to produce pictures and videos with nudity and sexually suggestive material, and it was earlier alleged to have been involved in creating inappropriate content of celebrities.
"The platform has been enabling the production and spread of these images for months without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying images of individuals in a graphic way".
In a statement, Ofcom said it was unlawful to "make or share non-consensual intimate images or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said networks such as X were mandated to take "necessary actions" to "lessen the risk" of users coming across unlawful content on their platforms and to take down it swiftly when they learn of it.
