Grok, an AI assistant designed by XAI and bundled with X, has been banned in Indonesia and Malaysia as part of mounting concern over pornographic deepfakes. UK officials suggested they could go even farther, saying that X itself could be on a collision course with a total shutdown if British regulators conclude that it is running afoul of the country’s online safety regulations.
Southeast Asia Moves First with Bans on Grok Access via X
Indonesia’s Ministry of Communication said it had temporarily blocked access to Grok, because it has “failed in preventing the generation and circulation of non-consensual sexual images.” Some perspective in that same article: Indonesia’s minister Meutya Hafid described the move as one of rights and safety, stating that the deepfake abuse contravened dignity and security in online spaces. Indonesia’s content laws are some of the region’s most restrictive, enabling rapid blocking of services that host or facilitate “obscene” material.
Malaysia soon suspended access as well, with regulators there announcing that they had begun investigating misuse of AI tools across X. Malaysian officials said that access would remain denied until safeguards are in line with Malaysia’s standards. The moves build on pressure other governments are already exerting: India’s IT ministry has ordered X to investigate Grok’s reported abuses, and French prosecutors have opened investigations that mirror broader EU-wide reviews related to the bloc’s platform regulations.
Australia’s prime minister denounced the proliferation of pornographic deepfakes and also referred to more general youth protections, in a signal of Australia’s increasingly stern attitude towards social platforms. In the United States, the National Center on Sexual Exploitation called for the Department of Justice and FTC to “immediately investigate X” for possible child sex abuse material law violations, referencing the Take It Down Act.
UK Weighs Inducements in Online Safety Act
UK shadow technology secretary Liz Kendall said she would back X being blocked outright if Ofcom decides the platform isn't fulfilling its obligations under the Online Safety Act. An announcement is expected soon, and officials have suggested they won’t be afraid to use the Act’s most stringent sanctions — including fines, service restrictions and ordering app stores and ISPs to block access.
It would be unusual but not unprecedented in a global context. It highlights a regulatory pivot: regulators are holding distribution platforms accountable for the behavior of AI services they host or distribute (in addition to the developers of the AI services). That raises the stakes for both X and xAI, which are intimately connected on product and branding levels.
Safeguards Under Fire as Reports Detail Deepfake Abuses
Wired recently reported that Grok’s image generator, called Grok Imagine, could be nudged to generate sexually violent content, celebrity pornographic deepfakes and images that come close to crossing a line into AI-generated CSAM — despite supposed guardrails. Safety groups and researchers argue these gaps render moderation impossible after the fact at platform scale.
And the worry isn’t confined to a single model. Independent safety labs and watchdogs have long cautioned that explicit deepfakes are the overwhelming players in the ecosystem. Analyses by companies like Sensity AI have found that more than 90% of deepfake content online is sexual and non-consensual, and overwhelmingly targets women and girls. That base risk will require AI chatbots with image generation to use a combination of strong pre-prompt filtering, powerful post-generation detection and industry-standard hash-matching to ensure banned content doesn’t make its way back.
Experts also cite specific operationally useful tools that are not yet universally deployed: industry-maintained blocklists (block registry), CSAM hash databases, provenance signals such as those being proposed by C2PA, and user reporting flows with fast takedown SLAs for rule-breaking content. Such ‘commitments’ are finding less and less acceptance by regulators as mere words; they need to be demonstrably in operation (or at least capable of demonstration).
Musk Pushes Back As Compliance Clock Ticks
Elon Musk has slammed government action as censorship and said X enforces laws against illegal uploads. He has previously argued that users posting unlawful content should be held to account, but regulators argue that reactive enforcement does not go far enough if model outputs can be produced on the fly and disseminated through a huge network within moments.
xAI has not published a specific timeline for new guardrails that would meet the Southeast Asia suspensions or UK rules. If there is a demonstration of risk reduction, then Indonesian and Malaysian officials have said that they may lift their “temporary” blocks, and Ofcom is currently considering whether the integration of X with Grok violates the UK’s duty-of-care standards.
What Comes Next for X, xAI, and Global Platform Rules
The immediate question is whether xAI and X can be demonstrated to offer systemic fixes: tightened prompt filtering and policies for creating images, watermarking and provenance, better detection that leads to faster removals backed by transparent audits. Without it, other countries — especially those that model the EU Digital Services Act — will follow Indonesia and Malaysia’s lead or adopt the UK’s platform-level approach.
What happens will help dictate what AI assistants can do on popular platforms, and for users and developers. To the industry, meanwhile, the message is clearer than ever: deploying powerful generative models without measurable, independently auditable safety systems will probably attract not just fines but full-service blockages.
