The Children's Commissioner for England, Dame Rachel de Souza, has issued a strong warning about the dangers posed by AI-powered apps that create deepfake images depicting the sexual abuse of children. Speaking to the Daily Mail, Dame Rachel highlighted rising concerns among teenage girls who fear that these apps could be used to digitally manipulate photographs of their faces onto pornographic images without their consent.
Dame Rachel’s recent report draws attention to the increasing availability of such technology on mainstream app stores, despite existing laws that prohibit creating or sharing sexually explicit images involving children. She underlined the significant risks posed to young people and called for swift government action to ban these AI tools.
"Children have told me they are frightened by the very idea of this technology even being available, let alone used," Dame Rachel said. She explained that young girls worry that anyone—a stranger, a classmate, or even a friend—could exploit these apps to produce naked images of them. Some girls have reportedly been avoiding posting photos or engaging on social media out of fear of becoming targets.
Addressing the issue directly, Dame Rachel stated, "We cannot sit back and allow these bespoke AI apps to have such a dangerous hold over children's lives." She emphasised that tools enabling the creation of deepfake sexual abuse images should not be permitted and urged the government to implement an outright ban. Additionally, she called for developers of generative AI technology to be held legally accountable if their products facilitate the creation of these harmful images. Dame Rachel also proposed that offences involving deepfake abuse be classified as sexual violence under the law.
The government responded by reaffirming its stance against the creation, possession, and distribution of child sexual abuse material, including AI-generated content. A spokesperson pointed to the protections under the Online Safety Act, which requires digital platforms to remove such illegal content or face substantial penalties.
This development brings to light the challenges of regulating emerging AI technologies and balancing innovation with the need to protect vulnerable groups, particularly children, from exploitation and harm online.
Source: Noah Wire Services