AI cannot prevent misdiagnoses

Up to 15 percent of all patients who seek medical treatment receive an incorrect diagnosis. Misdiagnosis is thus one of the most common and costly medical problems worldwide. Diagnosis is particularly challenging in emergency rooms, where a large number of patients with differing complaints are treated, often under great time pressure.

To reduce misdiagnoses, there is an increasing use of Computerized Diagnostic Decision Support Systems (CDDSS). By analysis of symptoms and findings, these systems are intended to increase diagnostic accuracy and to support medical professionals in making diagnoses. However, it is debatable whether diagnostic systems based on Artificial Intelligence (AI) actually improve diagnoses. So far, robust study data from clinical practice are scarce.

The first study worldwide on AI-based diagnostic systems in acute medicine

A research team led by the Department of Emergency Medicine has now investigated the effectiveness of AI-based diagnosis support in acute medicine in the first study worldwide on CDDSS. The study, the results of which have just been published in «The Lancet Digital Health», involved a total of 1204 patients who were treated for non-specific symptoms (such as fainting, abdominal pain or fever of unknown origin) in four Swiss emergency departments between June 2022 and June 2023. The participating emergency departments were divided into two alternating phases: during the intervention phases, the doctors used the AI-based system «Isabel Pro» to support them in making a diagnosis. In the control phases, the diagnoses were made without technical aids. The quality of the diagnosis was measured by whether, within 14 days of their treatment, patients required unplanned medical follow-up, diagnoses were changed retrospectively, unexpected intensive care admissions were required or whether there were any deaths.

No measurable advantage in AI-based diagnosis support

The results are surprising: a diagnostic quality risk occurred in 18 percent of patients, both in the phases with and without AI-based diagnostic support. And in terms of serious adverse events and of resource utilization, measured in Swiss francs, there was no difference between the groups. Despite optimized technology and extensive training of medical staff, the study was unable to demonstrate any relevant advantage in using CDDSS. «AI-based diagnosis support has no measurable effect for patients in emergency medicine, regardless of whether one considers medical, economic or procedural differences, » is the way Professor Wolf Hautz, Senior Consultant at the Department of Emergency Medicine and lead author of the study, sums up the results.

New impetus for research

The results of the study make it clear that computer-aided diagnostic systems, at least in their current stage of development, have no significant influence on diagnostic quality in emergency medicine. «The AI available at the moment will not solve the problem of misdiagnosis. We need to pursue other solutional approaches to improve diagnostic quality – and in particular to considerably intensify research in this area, which is currently in its infancy, » adds Professor Hautz.

To this end, the Swiss National Science Foundation (SNSF) is supporting the formation of a working group at the Department of Emergency Medicine of Inselspital on «Collaborative Decision-making». The current study was co-financed by the National Research Program «Digital Transformation» (NFP 77) of the SNSF.

Source: Insel Gruppe