A recent study published in Nature Medicine on February 5, 2025, explores the potential of chatbots in enhancing physician performance and answering complex clinical questions.

Led by Jonathan H. Chen, MD, PhD, the research team is investigating how chatbots can assist in clinical management reasoning, which involves nuanced decision-making based on various patient factors.

The study involved a comparative analysis of 46 doctors supported by chatbots, 46 doctors with only internet access, and the chatbot alone, evaluating their responses to five patient cases.

Results revealed that the chatbot alone outperformed doctors relying solely on internet searches, while physicians using chatbots matched the chatbot's performance.

This follows a previous study published in October 2024 in JAMA Network Open, which indicated that chatbots demonstrated higher diagnostic accuracy than doctors, even when doctors utilized chatbots.

The research highlights the improving capabilities of artificial intelligence-powered chatbots in diagnosing diseases, yet their effectiveness in addressing nuanced medical inquiries remains under investigation.

The findings raise important questions about the collaboration between physicians and chatbots and its implications for medical decision-making.

Chen emphasizes the potential benefits of integrating human expertise with computer capabilities to enhance medical decision-making.

Despite the advancements in AI, Chen advises against patients opting for chatbots over traditional doctor visits, underlining the need for credible information in the age of AI.

The research was conducted in collaboration with several institutions and received funding from various foundations and organizations.

Doctors' decisions were evaluated against a rubric developed by board-certified doctors to assess the appropriateness of their medical judgments.