Artificial intelligence is transforming societies at an unprecedented speed, bringing both new opportunities and new risks. Among the most concerning developments is the rise of deepfake technology; highly realistic fake images, videos and audio generated using AI. As deepfakes become more widespread and difficult to detect, they are increasingly being used to spread disinformation, manipulate public opinion and target individuals online.
What are deepfakes?
Deepfakes are synthetic media generated or modified by artificial intelligence. These can be images, audio, or video that imitate real people, making it appear as if they said or did things they never did.
UNESCO defines deepfakes as “digital forgeries” that are so realistic they can convincingly mimic a person’s voice or likeness. They blur the line between reality and fiction while becoming increasingly convincing, accessible and scalable.
How are deepfakes created?
Deepfakes are created using artificial intelligence systems trained on large datasets of images, videos or audio recordings. These systems learn to replicate human features such as faces, voices and movements, enabling the generation of highly realistic synthetic media.
Why are deepfakes dangerous?
Deepfakes not only create false realities, but they also “erode the very mechanisms by which societies construct shared understanding”. This deliberate manipulation contributes to a growing “crisis of knowing”. We are approaching a synthetic reality threshold where it is no longer possible to distinguish authentic from fabricated media without technological assistance. These developments pose a growing challenge to trust in digital information ecosystems.
Deepfakes can spread misinformation and are increasingly hard to detect, either by the human eye or by existing cybersecurity systems. Cybercriminals can, for example, exploit or imitate voices and video images to breach security barriers. By impersonating people in video or audio format, they can effectively steal their credentials. Such attacks can target automated systems that authenticate based on voice recognition.
In addition, manipulated content can reinforce societal biases and stereotypes, trigger gender-based violence, or escalate ethnic, religious, and political divisions. These risks may be further exacerbated in countries with low levels of digital literacy.
How is AI changing online abuse?
Deepfakes are also transforming the nature and scale of online abuse. The rise of powerful AI image generation tools has significantly simplified the production and dissemination of harmful content, including sexual abuse material. Deepfakes can be produced with relatively limited technical expertise and shared widely across platforms, increasing both the reach and impact of harmful material. As a result, digital abuse is becoming more scalable, targeted, particularly at women and children, and difficult to detect.
Deepfake abuse can include sharing private images without consent or creating AI-generated sexual content through morphing, splicing, or superimposing photographs and videos.
Disproportionate impact on women…
According to a survey, 38 per cent of women have experienced online violence, and 85 per cent have witnessed it. Many deepfake tools, often developed by male teams, are not even designed to work on images of men, highlighting the gendered nature of this technology.
Deepfake pornography accounts for 98 per cent of all deepfake videos online. Deepfake videos were estimated to be 550 per cent more prevalent in 2023 than in 2019. Once the content is online, it can be replicated endlessly, saved to private devices, and shared across platforms, making it nearly impossible to fully remove.
Anonymity is also contributing to the rise of digital violence against women. Importantly, online abuse does not remain confined to digital spaces and can quickly escalate into real-life harm.
… and children
At least 1.2 million children have disclosed having had their images manipulated into sexually explicit deepfakes, according to a 2025 study across 11 countries conducted by UNICEF, INTERPOL and ECPAT, a global network working to end the sexual exploitation of children worldwide. In some countries, this means one child per classroom. Most of them worry that AI could be used to create fake sexual images or videos. At the same time, AI-generated content fuels demand for abusive content and presents challenges for law enforcement in protecting children.
UNICEF also highlighted that these developments represent a significant escalation of risks children face in the digital environment and create new threats to children’s right to be protected from sexual abuse. Just a few years ago, high-quality generative models required significant computing power. Today, this type of content can be produced by anyone. Child sexual abuse material (CSAM) is also being produced and distributed at an alarming pace. This material often includes ‘deepfake’ nudes created in peer-to-peer contexts, disproportionately targeting girls. Such content normalises the sexualisation of children and can fuel demand. It also makes it harder to judge if an image depicts a real child in urgent need of help.
A justice gap
Deepfakes can have serious real-life consequences for individuals. Unfortunately, justice remains out of reach for most survivors, and underreporting is one of the biggest barriers to accountability. For some survivors, the justice system itself can become another source of trauma, with scrutiny focused more on the victim than the perpetrator. Prosecutions are rare, enforcement is lagging, and online platforms routinely fail to act. Fewer than half of countries have laws that address online abuse, and even fewer have legislation specifically covering AI-generated deepfake content.
1.8 billion women and girls still lack legal protection from online harassment and other forms of technology-facilitated abuse. Fewer than 40 per cent of countries have laws protecting women from cyber harassment or cyber stalking.
What needs to change?
Stronger international cooperation and more effective legal frameworks are needed to address AI-driven digital abuse.
Governments must adopt legislation with clear definitions of AI-generated abuse. States should further ensure sanctions for offences relating to child sexual exploitation. Accountability and transparency are essential.
At the same time, law enforcement needs training, resources and dedicated capacity to collect and preserve digital evidence. Trained, trauma-informed law enforcement and legal professionals, as well as free legal aid should be available.
Tech companies must be legally required to proactively monitor for and remove abusive content and face real financial consequences when they fail to act. Every actor in the AI value chain, from dataset providers to model developers, must embed safety-by-design.
Moreover, it is important to know the provenance of the data used to train AI models to maintain transparency and anticipate potential issues related to accuracy, bias, or licensing. Generative AI developers create training datasets using material from the web, often without clarity about the consent of copyright holders. The trained AI model will subsequently generate new content based on those materials, potentially infringing the copyright holder’s rights.
Schools, parents and caregivers also have a critical role to play. Parents and caregivers need to be informed about AI-enabled sexual exploitation and abuse, and have the knowledge to support affected children. Schools should educate students about AI-related risks such as deepfake nudes and ‘nudify’/’undress’ AI tools, online safety, and what to do when experiencing abuse. Children should be encouraged to report harmful behaviour.
What is the UN doing?
United Nations agencies are calling for safer AI design, greater platform accountability, better detection tools and stronger laws.
- UNESCO has developed ethical standards for artificial intelligence through its Recommendation on the Ethics of Artificial Intelligence.
- UNICEF has issued Guidance on AI and Children to lay out the foundations for child-centred AI.
- UN Women has warned that AI is accelerating technology-facilitated violence against women and continues to advocate for stronger protections against this type of violence.
- The International Telecommunication Union is working on global standards for watermarking, content authentication and digital verification to help users distinguish authentic from AI-generated media.
Further reading:
UNRIC Library Backgrounder: Artificial Intelligence
Deepfakes and the crisis of knowing
Artificial Intelligence and Child Sexual Abuse and Exploitation
Online safety 101: What every woman and girl should know
AI-powered online abuse: How AI is amplifying violence against women and what can stop it
Recommendation on the Ethics of Artificial Intelligence
Detecting deepfakes: Generative AI uptake casts doubt on multimedia content