The study, published in npj Digital Medicine by Jiazi Tian and colleagues, is titled An autonomous agentic workflow for clinical detection of cognitive concerns using large language models (2026).

Open-source tool Pythia is released alongside the study to enable other health systems to deploy autonomous prompt optimization for AI screening applications.

Calibration challenges were identified, including AI struggles with narratives lacking context and with documentation that lists cognitive concerns in problem lists rather than in narrative form, though performance remained strong in balanced tests.

The system analyzes over 3,300 notes from 200 anonymized Mass General Brigham patients to generate cognitive screening signals and flag individuals for formal assessment.

The work underscores the importance of early detection for cognitive impairment, especially with new Alzheimer's therapies that are most effective when started early.

The approach uses open-weight large language models and five autonomous agents that critique and refine each other’s reasoning in an iterative loop, reminiscent of a clinical case conference.

Authors stress transparency about calibration gaps to guide future improvements and to build trust in clinical AI, noting that cognitive impairment remains underdiagnosed and early detection is increasingly critical with new therapies.

All processing happens locally on hospital infrastructure, with no patient data leaving the institution, addressing data privacy concerns.

Mass General Brigham researchers unveiled one of the first fully autonomous AI systems that screens for cognitive impairment using routine clinical notes without human prompts after deployment, achieving 98% specificity in real-world testing.

The system employs a five-agent design where independent expert re-evaluation resolves disagreements between AI and human reviewers, providing defensible judgments as the expert validated the AI’s reasoning in a majority of disagreement cases.

In validation, the system showed 98% specificity, with 91% sensitivity in balanced testing and 62% sensitivity under real-world prevalence where 33% of cases are positive.

After deployment, the system runs without human intervention and uses five specialized agents that critique and refine each other’s reasoning in an iterative loop, forming a digital clinical team.