Dame Rachel de Souza urges the government to ban AI-driven apps that create deepfake images depicting child sexual abuse, raising concerns over the risks these technologies pose to children's safety and mental well-being online.
The Children's Commissioner for England, Dame Rachel de Souza, has issued a call for a ban on apps that use artificial intelligence (AI) to create deepfake images depicting the sexual abuse of children. In a recent report, Dame Rachel highlighted growing concerns among teenage girls who feel frightened by the availability of apps that can digitally manipulate their faces onto pornographic images without their consent.
Dame Rachel warned that these AI-driven technologies, which are increasingly accessible through mainstream app stores, pose "alarming risks" to young people, who may unwittingly become victims of such crimes. She expressed concern over the ease with which strangers, classmates, or even friends could exploit smartphones to manipulate images and create harmful content.
"Children have told me they are frightened by the very idea of this technology even being available, let alone used," Dame Rachel said. "They fear that anyone – a stranger, a classmate, or even a friend – could use a smartphone as a way of manipulating them by creating a naked image using these bespoke apps."
She also revealed that many girls have resorted to avoiding posting pictures or engaging online altogether to minimise the risk of being targeted by deepfake technology. "We cannot sit back and allow these bespoke AI apps to have such a dangerous hold over children's lives," Dame Rachel added.
While it is already illegal to create or share sexually explicit images of children, the Commissioner pointed out that the AI technology involved in generating such images is not currently against the law. As a result, she is urging the government to impose stronger legal responsibilities on developers of generative AI tools that enable the creation of deepfake abuse images. Additionally, she advocates for deepfake abuse to be formally recognised within the law as a form of sexual violence.
In response, the government stated: "Creating, possessing or distributing child sexual abuse material, including AI-generated images, is abhorrent and illegal. Under the Online Safety Act, platforms of all sizes now have to remove this kind of content, or face significant fines."
The discussion raises critical questions about the regulation of emerging AI technologies and their impact on the safety and well-being of children in digital spaces.
Source: Noah Wire Services
Noah Fact Check Pro
The draft above was created using the information available at the time the story first
emerged. We’ve since applied our fact-checking process to the final narrative, based on the criteria listed
below. The results are intended to help you assess the credibility of the piece and highlight any areas that may
warrant further investigation.
Freshness check
Score:
9
Notes:
The narrative refers to Dame Rachel de Souza as the current Children's Commissioner for England, a role she has held since 2021, and discusses very recent developments around AI-generated deepfake child abuse images. The call and government response appear timely and reflect ongoing regulatory discussions as of early 2025. No indication the content is recycled or outdated; it covers emerging technology and policy concerns still actively debated.
Quotes check
Score:
8
Notes:
Direct quotes from Dame Rachel de Souza are present, reflecting her recent statements on AI deepfake risks to children. These quotes match her known public communications on this issue. No earlier sources or press releases found with identical wording, suggesting these may be original or recent remarks rather than recycled quotes.
Source reliability
Score:
6
Notes:
The narrative originates from the Daily Mail, a widely read UK publication known for broad coverage but with mixed reputation on reliability and occasional sensationalism. Although it reports on verifiable public figures and official government statements, caution is advised as the outlet is less authoritative than established global news agencies.
Plausability check
Score:
9
Notes:
The claims about AI deepfake apps posing risks to children and calls for legal reforms are plausible and consistent with known technological capabilities and ongoing policy debates. The government's response echoes actual UK legal frameworks (Online Safety Act). No contradictory evidence found, though some details on AI app prevalence and specific legal technicalities cannot be independently verified here.
Overall assessment
Verdict (FAIL, OPEN, PASS): PASS
Confidence (LOW, MEDIUM, HIGH): HIGH
Summary:
The narrative is fresh and relevant to current AI and child protection issues, featuring credible quotes from a recognised official and consistent with official legal frameworks. Despite originating from a mixed-reputation outlet, the factual elements about legal calls, AI risks, and government response are plausible and supported by public knowledge, warranting a high confidence assessment.
