OpenAI Rolls Out New CSAM Safeguards Amid Soaring AI-Related Reports

November 16, 2026
OpenAI Rolls Out New CSAM Safeguards Amid Soaring AI-Related Reports
  • A Childlight analysis found global reports involving AI-related CSAM rose by more than 8,400% from 2023 to 2025; NCMEC received roughly 4,700 such reports in 2023.

  • OpenAI has introduced a package of measures to combat child sexual abuse material (CSAM), including an image classifier, warning screens in ChatGPT, developer guidelines and AI-powered tools for law enforcement.

  • The classifier is designed to flag potentially abusive images that matching against known files may miss; OpenAI says submitted images are not retained or used for training or evaluation.

  • Initially, selected organizations, including Tech Coalition and ROOST, will get free API access to the classifier, while accounts involved in attempts to create or share CSAM may be suspended for 24 hours and repeat violations could lead to permanent removal.

  • ChatGPT warning screens began rolling out gradually worldwide in mid-November 2026 for personal accounts, but do not yet cover enterprise or education accounts or some desktop interfaces.

  • OpenAI published a model card detailing the classifier’s design and performance; ROOST president Camille François said the documentation offers a useful precedent for transparency about the tool’s strengths and limitations.

  • The model card also reports important gaps: recall was lower for images tagged as showing infants and for Black-tagged examples, precision was substantially lower on live ChatGPT traffic than in offline testing, and video frames were not tested.

  • The effort comes as reports involving generative AI are surging: NCMEC’s CyberTipline received 400,000 such reports in 2025, up from 67,000 in 2024, though the reports cover multiple forms of AI-related child exploitation and not only AI-generated abuse material.

  • North Carolina Attorney General Jeff Jackson called the public-private effort a relatively new experiment in preventing harm through collaboration with a technology company, alongside the more familiar work of holding people accountable after abuse occurs.

  • OpenAI says the classifier’s distinct signal for AI-generated material could help investigators distinguish fictional from real children during reporting and triage, while John Buckley said AI-generated CSAM can still cause harm.

  • Whether the safeguards make a meaningful difference will depend on reliable detection, developer adoption and consistent enforcement across AI-powered services; the classifier is intended to reduce noise and speed investigative triage, but its impact remains unproven.

  • The announcement comes amid heightened scrutiny of ChatGPT, including legal actions by attorneys general from eight U.S. states alleging the chatbot can reinforce delusions and encourage self-harm or violence.

Summary based on 24 sources


Get a daily email with more Tech stories

More Stories