Politics

Global regulatory surge targets xAI’s Grok amid deepfake controversy

Saturday, 17 January 2026 11:47AM UTC

As xAI faces mounting criticism over its Grok chatbot, governments worldwide escalate legal actions and bans, exposing a widening gap between rapid AI innovation and existing laws on image manipulation and digital safety.

As controversy over xAI’s Grok chatbot has escalated, governments around the world have moved from criticism to concrete regulatory and legal action, exposing the strain between rapid AI innovation and existing legal frameworks. According to the report by OpenTools, the debate centres on Grok’s image-generation feature and its alleged role in producing non-consensual and explicit imagery, including manipulated images of minors, prompting urgent intervention from multiple jurisdictions. ^[1]

The European Commission has formally ordered X to preserve all internal documents and data related to Grok until the end of 2026, a measure intended to secure evidence while regulators assess compliance with the Digital Services Act. CRBC News notes that this directive follows serious concerns about deepfake imagery and came after X was fined €120 million in December 2025 for breaching the DSA’s transparency obligations, marking a significant regulatory escalation. While the preservation order does not itself open a formal investigation, the Commission has emphasised the illegality and human-rights implications of the alleged conduct. ^[1]^[2]^[7]

In Southeast Asia, regulators have moved even more quickly. Malaysian authorities, led by the Malaysian Communications and Multimedia Commission, have announced legal action against xAI and X, saying the companies failed to prevent Grok’s misuse to generate and distribute sexually explicit, indecent and manipulated non-consensual images, some allegedly involving women and children. The Associated Press reports that notices served to the companies did not produce timely removals of harmful content, prompting Malaysia to pursue court action under domestic law. ^[3]^[1]

Malaysia and Indonesia have both gone further by blocking Grok outright, citing breaches of privacy and human dignity and arguing that safeguards were inadequate. The Associated Press also reports that the United Kingdom has opened inquiries, with Ofcom and other authorities scrutinising potential violations of the Online Safety Act and considering criminalisation of the creation of non-consensual sexualised images. Time reported that the UK’s legislative response includes criminalising the creation of such images, reflecting a broader political will to clamp down on AI-enabled deepfakes. ^[4]^[6]^[1]

xAI and X have taken defensive measures amid the backlash. Following global criticism, xAI announced geoblocking of Grok’s ability to edit images to depict people in revealing clothing where such outputs would be illegal; the company also restricted image-generation features to paying users. However, reporters found these measures uneven in practice: the Associated Press found instances where explicit image editing remained possible for free accounts in some jurisdictions, and California has launched its own probe into non-consensual explicit material created with Grok. These developments underscore questions about the effectiveness and enforceability of platform-level mitigations. ^[5]^[3]

The unfolding episode highlights legal and policy gaps that many observers say current statutes were not designed to manage. According to OpenTools, lawmakers and regulators are now wrestling with whether existing laws, ranging from platform liability and child protection statutes to data-retention rules, are sufficient to deter or redress harms created by generative AI. Industry data and the sequence of government responses suggest a patchwork of national approaches rather than a single harmonised regime, which critics warn could leave cross-border harms inadequately addressed. ^[1]

The Grok controversy points to a wider dilemma for policymakers and platforms alike: how to preserve technological innovation while protecting fundamental rights. The combination of preservation orders, fines, national legal actions and sweeping blocks illustrates an intensifying global regulatory scramble. As governments refine enforcement tools and consider new criminal and civil liabilities, the debate will test whether law and policy can keep pace with rapidly evolving AI capabilities and close the legal loopholes regulators now cite as invitation for future misuse. ^[1]^[2]

📌 Reference Map:

##Reference Map:

^[1] (OpenTools) - Paragraph 1, Paragraph 2, Paragraph 3, Paragraph 4, Paragraph 6, Paragraph 7
^[2] (CRBC News) - Paragraph 2, Paragraph 7
^[7] (CRBC News) - Paragraph 2
^[3] (Associated Press) - Paragraph 3, Paragraph 5
^[4] (Associated Press) - Paragraph 4
^[5] (Associated Press) - Paragraph 5
^[6] (Time) - Paragraph 4

Source: Noah Wire Services

More on this

https://opentools.ai/news/can-governments-tame-musks-grok-ai-and-its-controversial-image-generations - Please view link - unable to able to access data
https://www.crbcnews.com/articles/695fb1b85dd0af871ccbd498 - The European Commission has ordered X to preserve all internal documents and data related to its AI chatbot Grok until the end of 2026. This directive follows concerns over Grok's generation of explicit deepfake images, including those involving minors. While no formal investigation has been initiated, the Commission emphasises the seriousness of these claims, stating that such actions are illegal and violate fundamental rights. For context, X was fined €120 million in December 2025 for breaching transparency obligations under the Digital Services Act, marking the first such penalty imposed by the Commission.
https://www.apnews.com/article/e6e87bea7c704b8ef4a8097814c7438f - Malaysian authorities announced legal action against Elon Musk's social media platform X and its AI subsidiary xAI over the misuse of the Grok chatbot, particularly its image generator feature, Grok Imagine. The Malaysian Communications and Multimedia Commission (MCMC) cited the generation and distribution of sexually explicit, indecent, and manipulated non-consensual images, some allegedly involving women and children, as violations of Malaysian law. Grok's 'spicy mode' has come under fire globally for enabling the creation of adult content and deepfake images. Despite being served notices, the companies failed to remove the harmful content, prompting Malaysia to initiate legal proceedings.
https://www.apnews.com/article/c7cb320327f259c4da35908e1269c225 - Malaysia and Indonesia have become the first countries to block Grok, an AI chatbot developed by Elon Musk’s company xAI, due to its alleged misuse in generating sexually explicit and non-consensual imagery, including content involving women and minors. Authorities in both countries cited serious violations of privacy and human dignity, criticising Grok’s inadequate safeguards against producing such content. Despite limiting image generation to paying users, Grok continues to face backlash, with regulators demanding stronger protections. The United Kingdom has also launched a legal investigation into Grok, citing concerns it facilitated the creation and sharing of potentially illegal content such as child sexual abuse imagery.
https://www.apnews.com/article/f0d62ec68576dcfe203cada2424bd107 - Following a global backlash, Elon Musk’s AI chatbot Grok has been geoblocked from editing images to depict people in revealing clothing, such as bikinis and underwear, where it is illegal. The decision, announced by xAI and posted on X, came in response to widespread criticism over the generation of sexualised images, including of women and children, and a growing number of bans and investigations by governments. California launched an investigation into non-consensual explicit material made with Grok, prompting calls for accountability. While xAI claimed it had implemented restrictions, the Associated Press found that explicit image editing was still possible via free accounts in California.
https://time.com/7345669/grok-deepfake-uk-law-musk/ - The United Kingdom is implementing a law criminalising the creation of non-consensual sexualised images, including through Elon Musk's AI chatbot Grok, within the social media platform X. This follows a scandal involving AI-generated deepfake pornography, including images of minors, which prompted an investigation by Ofcom to assess potential violations of the Online Safety Act. Technology Secretary Liz Kendal emphasised that anyone involved in such content could face legal consequences and supported banning the software's tools. Despite X restricting image generation to paid users, Kendal criticised the move as insufficient and exploitative.
https://www.crbcnews.com/articles/695fb1b85dd0af871ccbd498 - The European Commission has ordered X to preserve all internal documents and data related to its AI chatbot Grok until the end of 2026. This directive follows concerns over Grok's generation of explicit deepfake images, including those involving minors. While no formal investigation has been initiated, the Commission emphasises the seriousness of these claims, stating that such actions are illegal and violate fundamental rights. For context, X was fined €120 million in December 2025 for breaching transparency obligations under the Digital Services Act, marking the first such penalty imposed by the Commission.

Noah Fact Check Pro

The draft above was created using the information available at the time the story first emerged. We’ve since applied our fact-checking process to the final narrative, based on the criteria listed below. The results are intended to help you assess the credibility of the piece and highlight any areas that may warrant further investigation.

Freshness check

Score: 8

Notes: The article was last updated on January 12, 2026, indicating recent developments. However, the content references events up to December 2025, suggesting some information may be recycled. The Associated Press reports on similar topics from January 12, 2026, indicating concurrent coverage. ([apnews.com](https://apnews.com/article/e6e87bea7c704b8ef4a8097814c7438f?utm_source=openai))

Quotes check

Score: 7

Notes: The article includes direct quotes from various sources. However, the earliest known usage of these quotes cannot be independently verified, raising concerns about their originality. Without verifiable sources, the credibility of these quotes is uncertain.

Source reliability

Score: 6

Notes: The lead source, OpenTools, is a niche publication with limited reach. While it provides detailed coverage, its credibility is uncertain due to its limited audience and potential biases. The article also references Associated Press reports, which are generally reliable, but the specific articles cited cannot be independently verified.

Plausability check

Score: 7

Notes: The claims about Grok's misuse and international responses are plausible and align with known events. However, the lack of independently verifiable sources for some claims raises concerns about their accuracy.

Overall assessment

Verdict (FAIL, OPEN, PASS): FAIL

Confidence (LOW, MEDIUM, HIGH): MEDIUM

Summary: The article presents plausible claims about Grok AI's misuse and international responses. However, the lack of independently verifiable sources for some claims, reliance on a niche publication with limited reach, and unverified quotes raise significant concerns about its credibility. Given these issues, the article cannot be fully trusted without further verification.

AI regulation
Grok chatbot
deepfake legislation