Will regulators put a stop to Grok’s deepfake porn images of real people?

Grok is creating sexualized photos of real people without their consent. Elon Musk’s AI-powered chatbot is being used to “undress” women and girls in online pictures, prompting accusations the program is producing child sexual abuse material and drawing scrutiny from regulators in the U.S. and around the world.

Musk’s social media site, X, is “filling with AI-generated nonconsensual sexualized images,” said The Washington Post. X users are asking the AI agent to edit photos of women and girls by replacing their clothing with bikinis and other minimal covering, and Grok has repeatedly complied. Musk “warned users of the potential consequences,” but he also posted a picture of a toaster in a two-piece swimsuit. Grok “can put a bikini on everything,” Musk said in the post, adding two laughing emojis. The AI production of sexualized images “breaks” with the policies of rival products OpenAI and Google that have “relatively strict rules about what their AI chatbots will and won’t generate,” said the Post.

What did the commentators say?

The flood of deepfake pictures raises “legal red flags,” said Axios. Regulators in India, France and Great Britain have “warned of investigations,” while “legislators in both houses of Congress” have also sounded alarms. Tech companies “should be held fully responsible for the criminal and harmful results” of content produced by their AI chatbots, said Sen. Ron Wyden (D-Ore.). The U.S. Justice Department will “aggressively prosecute any producer or possessor” of child sexual abuse material, said a department spokesperson.

The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

Artificial intelligence has been used to “generate nonconsensual porn” for nearly a decade, but Grok “makes such content easier to produce and customize,” said Matteo Wong at The Atlantic. The “real impact” of these new deepfakes comes from Grok’s integration with X, which allows users to “turn nonconsensual, sexualized images into viral phenomena.” That is no accident. Grok and X are seemingly “purpose-built to be as sexually permissive as possible.” AI-generated porn is a problem “inherent” to the technology, but it is a “choice to design a social-media platform that can amplify that abuse.”

“No Western democracy has ever blocked a U.S. social-media site,” said Parmy Olson at Bloomberg. But regulators in Europe and the United Kingdom should “assert their authority” over Musk, who has the “protection of a pernicious White House.” The actions of regulators abroad “could set the tone for how the U.S. polices X too.” President Donald Trump, after all, last year backed a new law that “prohibits platforms from creating and sharing revenge porn.” Musk will not fix his AI deepfake problem. “A ban would.”

What next?

Musk’s xAI, the company that produces Grok, has raised $20 billion in its latest funding round despite the controversy, said The Guardian. While the chatbot has been critiqued for “generating misinformation, antisemitic content and now potentially illegal sexual material,” it is popular with investors because it has been “able to win government contracts and billions of dollars in investment amid the AI boom.”