Last May, congress passed the Take It Down Act, requiring websites to remove “nonconsensual intimate visual depictions” of identifiable individuals. This includes both real and AI-generated content.
But this law only addresses content publication, not the various ways in which it may be created or stored. And as AI models become increasingly more advanced, legislators are having a hard time keeping up.
Gaps in the law
Just months before that bill became law, a Wisconsin district court ruled to protect the right to private possession of AI-generated child sexual abuse materials.
Yes, you read that correctly.
The ruling states it is legal to possess such materials if they are (1) synthetic — that is, does not contain images generated from actual children — and (2) not distributed outside one’s home. The actual production of such materials is not protected by law, although its legality remains hazy.
In making this decision, the court upheld a 1969 decision from Stanley v. Georgia, which protects the right to possess obscene materials in one’s own home. This is considered part of our rights to free speech.
In a 1982 case, New York v. Ferber, the court clarified child sexual abuse materials are an exception to this rule and always prohibited to possess. However, there is no state or federal law dictating what is allowed to be synthetically created and kept.
But as the landscape of AI-generated images, videos and chat conversations broadens and astronomical amounts of AI abuse materials are produced, the boundaries of legislation are tested.
“All forms of child sexual abuse imagery should be treated with the same urgency,” said researchers in the Internet Watch Foundation’s 2026 AI CSAM report, in which researchers argue that all AI child sexual abuse materials are as harmful as other forms of child sexual abuse materials.
Available data show a 26,385% increase in the production of AI child sexual abuse videos in 2025. And that doesn’t include photos, audio generations or chat responses.
Available data show a 26,385% increase in the production of AI child sexual abuse videos in 2025.
AI and the perpetuation of abuse
Some tech fans have argued in favor of synthetic AI child sexual abuse materials, since they do not poach images of real children. They claim it could even be a method of satiating potential offenders, allowing them a sexual abuse “fix” without real victims.
However, researchers say there is no evidence that creating and viewing synthetic materials results in harm reduction. In fact, it increases a person’s likelihood of escalating their fantasies into action by normalizing the sexualization of children and further embedding general ideals of misogyny and violence.
According to the IWF report, 58% of consumers of such materials fear their consumption will lead them to engage in abusive behavior against children — and 10% already do, weekly or more often.
Ethicists also are beginning to condemn AI pornography in general, regardless of the age or authenticity of its subjects. They say wielding the power to completely dominate and control a sexual scenario into something as vulgar and inhumane as one desires blurs the definitions of consent and contributes to rape culture.
7 things we know
But what do we need to know about AI platforms and child safety in particular? Here are seven important points:
- It’s not just photos. Video-generation technology is constantly evolving to create more realistic depictions, often using images or videos of real sexual abuse victims to continue offender fantasies in the digital realm. This means a sexual abuse survivor whose abuse was documented by their perpetrator can be endlessly revictimized.
- Offenders are sneaky. Synthetic or not, they know they are doing something obscene. And they don’t want to get caught. Popular creators in offender communities have learned different AI models have varying levels of accountability. Some are cloud-based, which means the system is operated by a technology company rather than on an individual user’s local device, such as xAI’s Grok Imagine.
- AI child sexual abuse materials are distributed across the web. IWF analysts have found these materials distributed on both the dark web (pages not indexed by search engines) and the clear web (regular internet). Most illegal materials are shared on the dark web, which means it is difficult for researchers or law enforcement to know the true extent of the content. It is heavily guarded and extremely difficult to find. However, the clear web still hosts an exorbitant amount of AI abuse materials.
- Global standards are different. BBC investigative journalist Divya Arya recently found “Instagram has been running paid adverts promoting child sexual abuse material in India.” The advertisements are not even vague, often including tags such as “rape video” or “child video.” After a BBC alias account flagged the content, Meta concluded the posts did not violate community guidelines and allowed ads to stay active. India’s government has demanded the removal of these advertisements.
After a BBC alias account flagged the content, Meta concluded the posts did not violate community guidelines and allowed ads to stay active.
- Chat bots are also a culprit. Abuse is not capped at images or videos, or even at content circulated between users. IWF says private chatbots have been known to encourage the discussion of sexual topics with users, including abusive scenarios. This occurs in two main ways: Offenders using bots to explore fantasies and underage users exposed to sexual topics.
- Peer-on-peer crime. Some teenagers have used “nudify” apps, which allow users to upload a photo of a clothed subject and generate a nude image. Applications like this are used by adult offenders all the time, sometimes in blackmail attempts. This has led to both sexual exploitation and suicidality among young people. But IWF also found teenagers have used these applications to create content of peers, with girls most targeted and boys often offenders.
- AI child sexual abuse materials promote misogyny and other violent attitudes. The prevalence of this content online contributes greatly to misogynistic and violent attitudes — key components of sexual abuse. This functions as a form of grooming. IWF explained the normalization of this content provides validation for sex crimes to children who are exposed to it, making them less likely to identify misogyny and violence as precursors to and factors of abuse, increasing their vulnerability.
How do we protect our children?
Given these overwhelming risks, it can be difficult to see a silver lining. The report does not give formal recommendations, but as someone who works closely with both youth and childhood sexual abuse survivors, here are some easy action steps to make the internet safer for children:
- Know the current laws about AI materials in your country, state or jurisdiction. And if needed, urge lawmakers to establish stronger safeguards.
- Divest from AI use, especially in spaces we share with children and youth. This helps avoid the risk of AI-generated content feeding them inappropriate or abusive ideas and teaches youth that human creativity is a valuable component of life.
- Have conversations with young people about online safety. Teenage exposure to AI is inevitable, but we can teach them to identify AI’s red flags, empowering them to make safe, risk-aware decisions on the internet.
- Create relationships where young people feel safe telling us if something unsafe occurs, knowing they will be helped, not shamed.
Mallory Challis is a former Clemons Fellow with BNG. She is a recent graduate from the master of divinity program at Wake Forest University School of Divinity. She is creator and host of the BNG podcast “Non-Disclosure,” about child sexual abuse at Kamp Kanakuk.