AI language models can match or exceed mid-career ophthalmologists on specific binary triage tasks in refractive surgery, yet still require human oversight for complex, multi-option decisions, serving as a scalable, affordable pre-screening layer to augment surgical decision-making.

Ground truth was established by a panel of three senior refractive surgeons with high agreement, who assigned recommendation scores and procedure classifications for each case.

Among the five models tested, DeepSeek-Chat offered the lowest cost per query and fastest response times while maintaining strong accuracy, whereas GPT-4o was the most expensive.

The study provides a rigorous benchmarking blueprint—structured expert-mimicking prompts, a large real-world dataset, multi-surgeon gold standards, and multi-metric evaluation—and discusses data privacy, liability, and integration into clinical workflows.

Models showed strong correlations with expert scores beyond binary judgments, mirroring gradations of suitability scores for different patients.

In the four-procedure multiclass task, agreement with the expert panel was more modest (for example, Qwen-Max reached Cohen’s kappa up to 0.743), underscoring AI as decision-support rather than autonomous in nuanced cases.

The deployment view emphasizes using AI to flag candidates and support decisions, with surgeons retaining final judgment, especially in borderline scenarios.

Top models achieved over 98.5% accuracy and an AUC above 0.96 for binary decisions on LASIK or SMILE suitability, at times outperforming the mid-career physician on straightforward judgments.

A Chinese ophthalmology team evaluated five leading large language models on real-world preoperative data from nearly 12,000 cases to assess suitability for refractive surgery options, benchmarking AI against an intermediate-level ophthalmologist.