Microsoft develops AI tool that can diagnose ’sequentially’ like a doctor
Microsoft says MAI-DxO outperformed physicians on simulated diagnostic evaluations; real-world clinical testing remains ahead.
TL;DR
- Microsoft reported 83% accuracy for MAI-DxO on complex New England Journal of Medicine cases, compared with 36% for physicians using reference materials.
- On emergency-department cases from MIMIC-IV-ED, the system scored 80% against 47% for physicians, according to Microsoft.
- The results are from simulated evaluations, and Microsoft says a prospective hospital study is planned.
Microsoft said its MAI-DxO system reached 83% diagnostic accuracy on complex cases drawn from the New England Journal of Medicine. The company compared that result with 36% for physicians who could consult reference materials and 22% for physicians without those resources. [1] [2]
The system uses an orchestrator to coordinate several AI agents through a sequence of questions, examinations and tests, rather than asking one model to produce an immediate answer from a complete case description. Microsoft also reported 80% accuracy on emergency-department cases in the MIMIC-IV-ED dataset, versus 47% for physicians. [1] [2]
Microsoft published the work in Science and said it is making parts of its Sequential Diagnosis Benchmark available for researchers to reproduce and compare results. Reuters described the evaluation as a study of complex diagnostic cases and noted that the system chooses questions and tests as it proceeds. [1] [2]
The comparison is based on benchmark cases, not a prospective trial showing improved outcomes for patients in routine care. Microsoft said it plans to evaluate the system with Beth Israel Deaconess Medical Center, leaving performance across hospitals, patient safety and clinical outcomes open questions. [1] [2]
Why it matters
The results put agent coordination and sequential reasoning at the center of a high-profile AI capability claim. For healthcare institutions, the gap between benchmark performance and prospective clinical evidence remains the key unresolved step.
Editor's note
Company-reported benchmark results published in Science; figures are from Microsoft and Reuters coverage. Simulation results do not establish real-world clinical benefit.