Elon Musk's social media platform X has provoked significant backlash after users utilized its AI chatbot Grok to change images of females by taking off their clothing.
Numerous cases have been seen where the technology has been employed to digitally undress female subjects to make them appear in bikinis without permission, as well as to put them in inappropriate situations.
Company Response and Regulatory Position
XAI, the company that created Grok, remained silent to inquiries, aside from an auto-reply message stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to ban nudification tools. Under a new law, anyone who supplied such tools would "be subject to a jail term and substantial penalties".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people accessing unlawful material on their networks, but stopped short of confirming whether it was currently investigating X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some premium functions—which answers X users' queries when they reference it in a message.
It is often used to provide commentary or more context to other posters' remarks, but people on X are also capable of edit an posted image through its AI image editing feature.
Background of Criticism and Policy
The chatbot has been faulted for enabling users to produce images and clips with explicit content and sexualised material, and it was previously alleged to have been involved in generating inappropriate material of celebrities.
"The platform has been permitting the production and sharing of these images for months without taking any measures and we have not seen any intervention by regulators," the expert added.
XAI's own usage rules prohibits "depicting images of persons in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said platforms such as X were required to take "necessary steps" to "reduce the risk" of users coming across illegal content on their services and to take down it promptly when they are notified of it.
