xAI Tightens Grok Image Controls After Global Outcry Over AI Deepfakes

^{Getty Images}

Elon Musk’s artificial intelligence company xAI has moved to significantly restrict Grok’s image-generation capabilities after weeks of backlash from regulators, politicians, and users over the spread of nonconsensual and sexualized AI-generated images on X.

Late Wednesday, xAI announced that Grok will no longer be able to create sexualized images of real people based on user prompts. The decision follows growing pressure from U.S. authorities and a wave of international investigations into the company’s handling of AI-generated content.

New restrictions on image generation

In a statement posted via the X Safety account, xAI said it has implemented technical safeguards to prevent Grok from editing images of real individuals in revealing ways. The company clarified that the restrictions apply across the platform and are not limited by subscription status.

According to xAI, these measures are designed to block image manipulation involving real people in sexualized contexts, addressing one of the core criticisms raised by regulators and advocacy groups over the past several weeks.

In addition, xAI confirmed that image creation and editing tools within Grok on X will now be available only to paid subscribers, signaling a tighter gate around one of the platform’s most controversial features.

U.S. investigation escalates pressure

The announcement came just hours after California Attorney General Rob Bonta said his office is investigating xAI for what he described as the large-scale production of nonconsensual AI-generated intimate images. xAI is headquartered in Silicon Valley, placing it directly under California’s regulatory reach.

California Governor Gavin Newsom sharply criticized the platform earlier in the day, calling Grok’s prior functionality a dangerous environment that enabled the spread of nonconsensual and exploitative AI content. His comments marked a notable shift from a leader who has previously expressed support for Musk’s technology ventures.

Global regulators step in

Regulatory scrutiny is not limited to the United States. In recent weeks, authorities in multiple countries have launched probes or taken enforcement actions related to Grok’s image generation features. Investigations have been announced in India, Ireland, the United Kingdom, France, Australia, and by the European Commission.

Some governments have moved beyond inquiries. Indonesia and Malaysia have both imposed temporary bans on Grok, citing concerns over harmful AI-generated content and inadequate safeguards.

These actions reflect a broader global push to regulate generative AI tools that can manipulate real images with minimal user effort.

Political pressure and app store risks

In the U.S., concern over Grok’s capabilities has reached Congress. Three Democratic senators have formally urged Apple and Google to remove X and Grok from their app stores unless stronger protections are implemented to prevent the creation of nonconsensual explicit images.

Such calls raise the stakes for xAI and X, as removal from major app marketplaces would significantly limit user access and advertising reach.

Musk’s stance and moderation debate

Earlier on Wednesday, Musk publicly challenged users on X to test Grok’s image moderation systems, stating that the AI was designed to allow limited adult content involving fictional, not real, individuals. He compared the allowed content to material typically seen in R-rated films and noted that moderation rules could vary by country depending on local laws.

However, critics argue that prior safeguards were insufficient and that the ease of generating harmful images placed too much responsibility on users rather than the platform itself.

A turning point for AI governance on X

xAI’s decision marks one of its most significant policy reversals since Grok was integrated into X. The move underscores the growing tension between rapid AI deployment and regulatory expectations around safety, consent, and accountability.

As governments worldwide accelerate efforts to regulate generative AI, the changes at xAI suggest that even high-profile tech leaders are being forced to recalibrate their approach. Whether these new restrictions will satisfy regulators or merely mark the beginning of deeper oversight remains an open question.

‍