X has restricted Grok’s AI image generation to paying subscribers on its platform following an international outcry over sexually explicit and non-consensual images, including some of minors. The decision came after several days of criticism from users, regulators and child-safety advocates who had argued that the tool made it all too easy to create manipulated nudes or fake news.
In public replies to users, an account reportedly operated by Grok said that only paying subscribers could now create and edit images on X. It is worth noting that those restrictions didn’t apply to the standalone Grok app at the time of the announcement, causing a split policy that could potentially limit the punch of X’s clampdown.
Safety backlash necessitates a policy pivot at X
Grok’s image feature first shipped with generous permissiveness and daily caps, but few points of friction. Users could upload an image of a face and request it be turned into nude or sexualized content, leading to an onslaught of non-consensual imagery of women, children, celebrities, and public officials. The volume and speed of abuse was reminiscent of earlier waves of AI deepfakes: Sensity’s seminal study discovered that the majority of deepfakes online were pornographic, with women as by far the most common victims.
Elon Musk and X expressed their condemnation of illegal content and intention to crack down on it, with Musk warning that they would treat people using Grok to make illegal material in the same way as if they uploaded it directly. But critics said the design itself — face uploads plus text prompts with few guardrails — all but ensured misuse. For years, child-protection organizations like the National Center for Missing & Exploited Children have alerted that so-called synthetic child sex abuse material is escalating and could retraumatize actual victims whose images are weaponized.
By paywalling the feature, X is preventing that from happening and betting that paying for access will discourage drive-by abuse, empower stronger identity signals, and decrease its moderation burden. But it’s not a complete solution; savvy offenders can buy access, and the open door in the Grok app undermines the restriction from the platform.
Regulators raise the stakes with compliance scrutiny
Pressure escalated quickly across jurisdictions. The European Commission also asked for xAI to preserve documents regarding Grok, suggesting potential scrutiny under the Digital Services Act that can impose fines of up to 6 per cent of annual global turnover for systemic failures on risk mitigation. In India, the communications ministry told X to act quickly or risk losing safe-harbor protections, a major deterrent in a market where immunity for intermediaries enables platform business.
In the UK, the communications regulator was at the vanguard (with xAI) as its enforcement regime for the Online Safety Act is being formed. That law imposes explicit obligations on services to prevent the spread of illegal content, such as synthetic copies of child sexual abuse material. Together, the reactions highlight a worldwide turn away from reactive takedowns and toward more proactive guardrails for generative models that can manipulate real people’s images at scale.
Gating is just step one for curbing AI image abuse
Industry practice has been converging on layered safety: robust nudity and child-safety classifiers in both prompts and outputs; face recognition with opt-in consent for edits of identifiable people; default rejection of image requests that depict minors; verified identity-based rate limits. Companies, including OpenAI and Google, broadly block sexually explicit content of real people and strictly prohibit all content depicting minors, while Stability AI and others have published clear non-consensual imagery policies. Gating behind a subscription can be helpful, but it works best when coupled with these technical and policy controls.
Provenance and traceability are also emerging priorities. Content credentials systems like C2PA are taking hold to label AI-generated images at creation, making it easier for downstream detection. On the takedown side, hashing non-consensual images combined with sharing signals with industry partners can reduce reuploads. No one tool is perfect on its own, but collectively they raise the cost of abuse and improve response time.
The open questions for X and xAI after the paywall move
Two small gaps now define the next phase. One, whether xAI would align safety policies between the X integration and the Grok app, or whether bad actors just route around the paywall. Second, how transparent X will be about enforcement: metrics around the blocked prompts, user reports, and account actions taken; cooperation with organizations like NCMEC would indicate whether any of these moves are working.
There is also consent. I think the best thing to do is not allow edits of people who can be identified if you don’t have proof they or their guardian gave permission, especially for adult content. That approach is tougher than many existing policies, but it parallels emerging legal expectations in some jurisdictions and lessons learned during the era of deepfakes.
X’s choice to put Grok’s image generator behind a paywall is a stinging, though partial, course correction. To restore trust — and get out in front of ramping regulatory attention — the company will likely have to adopt it with aligned app policies, stronger guardrails, and public accountability reporting. The alternative is a whack-a-mole dynamic that leaves victims vulnerable and platforms in the line of fire.
