Money and business in the Middle East.

AI

OpenAI releases free CSAM detection tool, cuts review workload 48%

OpenAI is releasing an image classifier that detects child sexual abuse material, including AI-generated images.

· Last verified: 17 Nov 2026 (OpenAI)

Summary

  • OpenAI is releasing an image classifier that detects child sexual abuse material, including AI-generated images.
  • Vetted organizations get access free through the OpenAI API, with the National Center for Missing and Exploited Children assisting development.
  • The company says the tool flags 48% fewer images while finding roughly the same number of abusive ones.

The latest

A new image classifier built to detect child sexual abuse material, including AI-generated imagery, is being released free to vetted organizations through the OpenAI API. OpenAI said the tool returns scores that help reviewers triage which images need attention first, and that it is the first widely available CSAM classifier carrying a specific signal on whether material may be AI-generated. New ChatGPT warnings accompany the release.

Details

  • The tool: OpenAI said the classifier looks for visual signs of abuse rather than matching against known images, allowing teams to identify novel material not seen before. It assigns scores to prioritize review and flags whether an image may have been AI-generated — a signal the company said no other broadly available CSAM classifier provides.
  • The numbers: In an evaluation of user-uploaded images, OpenAI compared the classifier against its existing protections and estimated it would flag 48% fewer images while surfacing roughly the same volume of abusive content. Images the new classifier flagged were about twice as likely to actually contain abuse, according to the company.
  • Partner testing: OpenAI said early access partners ran their own evaluations. At fal, reviewers confirmed 72% of high-confidence flags as novel CSAM in production images. invideo AI reported one in four flagged cases ended in a ban. SafetyKit found the tool detected 90% of cases involving drawings or animations that sexualize children.
  • Who built it: The classifier was trained, evaluated and designed for real-world deployment with assistance from the National Center for Missing and Exploited Children, the nonprofit that operates the CyberTipline, a centralized hub for reporting online child exploitation in the United States.
  • Distribution: Access is free to vetted organizations through the OpenAI API, which the company said puts detection within reach of child safety groups and industry teams of varying sizes. OpenAI engaged the Tech Coalition and ROOST to help distribute the tool, and said it also supports rival systems including Thorn's Safer.
  • Cross-platform signals: OpenAI said it recently joined the Tech Coalition's Lantern program, under which participating companies share and receive signals relating to online child sexual exploitation and abuse. The stated aim is to strengthen collaboration across platforms so offenders moving between services can be tracked.
  • ChatGPT warnings: OpenAI is adding product warnings in ChatGPT that interrupt attempts to find abusive material, explain the consequences, and direct users toward help. The company said the warning states explicitly that sexualizing children causes harm, including when the requested material is AI-generated.
  • Enforcement: OpenAI said a flagged conversation will be removed and the user's ChatGPT access paused for 24 hours, with specialist support kept available. Repeated violations may bring longer restrictions, account closure, or reports to authorities. The company already prohibits CSAM and trains its models to refuse such requests.
  • Access path: OpenAI published a model card with further technical information and said organizations seeking access will be able to apply through an intake form. The company did not set out eligibility criteria for vetting applicants or a timeline for approvals.

Background

OpenAI said the release builds on its Child Safety Blueprint and existing work to detect, block and report abuse. Traditional detection relies heavily on matching against databases of known material, a method that struggles with newly generated images — the gap the company says the classifier targets.

Between the lines

The AI-generated signal is the part of the release aimed at OpenAI's own industry. Image generators have made novel abusive material cheap to produce, which breaks hash-matching systems built for known files. Offering the classifier free through the API, and backing competing tools like Thorn's Safer, positions OpenAI as supplying infrastructure to the sector rather than only policing its own products.

What's next

Watch whether the intake form opens broadly and which organizations are vetted for access, how partner evaluations hold up at larger scale, and what signals OpenAI begins contributing to the Tech Coalition's Lantern program.

Source: OpenAI