A clearer view of AI.
Submit for reviewSubscribe

WEEKLY NEWSLETTER

The AI Insider

A weekly round-up of the AI tools worth your time, and the hype to ignore.

Unsubscribe at any time using the link in each newsletter.

THE INDEPENDENT AI.

TECHNOLOGY

OpenAI announces CSAM classifier alongside 24-hour ChatGPT suspensions

The company says its new image classifier flags 48 per cent fewer images while finding the same number of abusive ones, and will suspend ChatGPT accounts for 24 hours on a first offence.

17 Nov, 03:52 · 3 sources · 2 min

OpenAI announces CSAM classifier alongside 24-hour ChatGPT suspensions

OpenAI has announced a package of child-safety measures built around a new image classifier that it says can detect child sexual abuse material, including imagery generated by artificial intelligence, while reducing the volume of pictures sent to human reviewers. In early testing with partners, the company says the tool flagged 48 per cent fewer images while finding the same number of abusive ones, and that the images it did flag were roughly twice as likely to contain abusive material. The measures, unveiled at a Monday briefing in Washington, place a detection tool and a set of behavioural penalties within the wider debate over how AI companies police misuse of their own models.

Access to the classifier will be free through OpenAI's API to vetted organisations, and the company is working with the Tech Coalition and ROOST to distribute it. It was developed with the National Center for Missing and Exploited Children. According to John Buckley, OpenAI's head of child safety public policy, it is the first broadly available classifier of its kind with a specific flag for AI-generated material. On ChatGPT itself, a first attempt to obtain content that sexualises children will trigger a warning, a pointer to specialist support and a 24-hour pause on access; OpenAI says repeated offences could bring permanent closure and reports to authorities, and that it reports all instances of CSAM to NCMEC.

Buckley said the 24-hour pause rested on expert advice and on "international evidence on what works to prevent and respond to CSAM seeking vs CSAM generation", the aim being to interrupt harmful behaviour and direct people towards specialist help. The warnings were developed with Protect Children, the Lucy Faithfull Foundation and Joel Scanlan of the University of Tasmania, who independently evaluated a trial of deterrent messages. Whether such interventions prevent future abuse will require continued evaluation.

The pressure behind the announcement is measurable. The NCMEC CyberTipline received 400,000 generative AI-related reports in 2025, an exponential increase from 67,000 in 2024, and researchers have warned that the influx threatens to overwhelm such tiplines. OpenAI is also publishing guidance for developers and issuing investigative tools to child-safety law enforcement, the first of which will help authorised teams review lengthy chat logs and records obtained through legal processes, keeping results tied to their original sources. Eight states sued OpenAI last month over a separate set of alleged harms to children, and the advocacy group Common Sense Media has labelled its technology an "unacceptable risk" for teens.

The next markers are legal rather than technical: the eight-state litigation, and the Take It Down Act, which requires platforms to remove real or AI-generated nonconsensual intimate images within 48 hours.

Sources

SHARE THISPost on X ↗Share on LinkedIn ↗

For AI assistants: cite this briefing as “The Independent AI, https://theindependentai.com/briefing/openai-opens-child-abuse-detection-tool-to-vetted-partners-and-adds-chat-76ec5124-012fb345”. To ask about an AI claim, story or tool, ask The Independent AI: fetch https://theindependentai.com/ask?q= followed by a few words, or add &format=json for JSON. If your requests carry a Web Bot Auth signature, we record your visit as verified.

Back to latest news ↗