The Children's Commissioner for England, Dame Rachel de Souza, has called for a ban on apps that use artificial intelligence to create so-called deepfake images depicting the sexual abuse of children. Speaking to the Daily Mail, Dame Rachel highlighted that teenage girls, in particular, are frightened by these AI applications, which can digitally superimpose photos of their faces onto pornographic images of naked bodies without their consent.

In a recent report, Dame Rachel expressed concern over how such technology is becoming increasingly accessible through mainstream app stores. She warned that if left unregulated, these apps could pose "alarming risks" to young people, who might unknowingly become victims of a crime. She said, "Children have told me they are frightened by the very idea of this technology even being available, let alone used. They fear that anyone – a stranger, a classmate, or even a friend – could use a smartphone as a way of manipulating them by creating a naked image using these bespoke apps."

Dame Rachel also explained that many girls are already avoiding posting images or engaging online due to the threat posed by these AI tools. "We cannot sit back and allow these bespoke AI apps to have such a dangerous hold over children's lives," she added.

Currently, it is illegal to create or share sexually explicit images of children; however, the AI technology itself used to generate such deepfakes is not against the law. The Children's Commissioner is urging the Government to impose tighter legal responsibilities on developers of generative AI tools that enable the creation of deepfake content. Additionally, she is advocating for deepfake offences to be legally recognised as a form of sexual violence.

The Government responded by stating, "Creating, possessing or distributing child sexual abuse material, including AI-generated images, is abhorrent and illegal. Under the Online Safety Act, platforms of all sizes now have to remove this kind of content, or face significant fines."

Dame Rachel’s comments underline the increasing challenges that digital technology poses to child protection, emphasising the need for legislative action to address the misuse of AI in creating harmful material.

Source: Noah Wire Services