It advances beyond prior benchmarks by reflecting real-world clinical workflows and diverse data sources through its design that spans multiple languages, specialties, and tasks.

Evaluation methods include chain-of-thought prompting and few-shot learning to simulate complex medical reasoning scenarios.

The research team features co-first authors Jiageng Wu and Bowen Gu, senior authors Jie Yang and Joshua Lin, and emphasizes interdisciplinary collaboration across pharmacoepidemiology, pharmacoeconomics, clinical medicine, and computational modeling.

Open-source LLMs sometimes match or exceed proprietary models, highlighting potential for democratized access to clinical AI tools.

Core themes include medical reasoning, diagnostic support, patient communication, clinical text summarization, and health equity as they relate to LLMs.

Evaluation and benchmarks cover medical tasks such as diagnosis, imaging reports, medical coding, and discharge summaries across various benchmarks and tasks.

BRIDGE maintains a continuously updated leaderboard on Hugging Face to track models and datasets, encouraging ongoing benchmarking and progress.

The benchmark represents 14 clinical specialties to ensure assessments reflect specialized knowledge and varied documentation styles.

The study evaluates 95 LLMs from 59 clinical AI initiatives across 14 specialties, using multiple inference strategies to simulate real deployment conditions.

BRIDGE is a comprehensive multilingual, multi-specialty benchmark from Mass General Brigham that evaluates how well large language models understand real-world clinical texts, including EHRs, case reports, and patient-doctor conversations across nine languages.

Notable findings across the field show that adapted LLMs can outperform experts in clinical text summarization, health systems are increasingly using LLMs as prediction engines, and GPT-4’s impact on physician performance comes with potential biases.

Funding for BRIDGE comes from the Patient-Centered Outcomes Research Institute, the NIH, and institutional scholarships, with a strong emphasis on transparency and ethical research through disclosures and compliance.