AI Medical Devices: Are We Using Them Before We Know They Work?

Artificial intelligence is increasingly making its way into healthcare, helping doctors interpret scans, assess cardiovascular risk, plan surgery, and make clinical decisions.

But how much do we actually know about whether these AI systems make patients healthier? A new analysis published in PLOS Digital Health looked at 1,357 AI-based medical devices authorised by the US Food and Drug Administration.

Just 34 had been included in registered clinical trials and only three had been tested to see whether they actually improved patient outcomes, such as survival, strokes, hospitalisation or quality of life, such as survival.

such as survival, stroke risk, hospitalisations or quality of life.

That raises a pretty fundamental question.

When an AI medical device is cleared for use, what exactly have we proved about it? The researchers also found that many of the studies that do exist were carried out in highly resourced healthcare systems and excluded groups including people over 75, pregnant women and non-English speakers.

And that matters beyond the US.

Countries around the world may rely on regulatory decisions made elsewhere, potentially making patients in lower-income countries inadvertent test populations for AI that hasn't been properly evaluated in their healthcare systems or communities.

So as AI becomes part of everyday medical care, should the rules change to make developers prove not just that their technology works, but that it actually makes a difference to patients? I spoke with Nura Issa from the Mubara University of Science and Technology in Uganda, Sebastian Ordonez, researcher in AI at MIT, and Leo Antonicelli, senior research scientist at MIT and associate professor at Harvard Medical School.

I started by asking Leo what does it mean when something is cleared by the FDA?

Artificial intelligence is increasingly making its way into healthcare, helping doctors interpret scans, assess cardiovascular risk, plan surgery, and make clinical decisions.

But how much do we actually know about whether these AI systems make patients healthier? A new analysis published in PLOS Digital Health looked at 1,357 AI-based medical devices authorised by the US Food and Drug Administration.

Just 34 had been included in registered clinical trials and only three had been tested to see whether they actually improved patient outcomes, such as survival, strokes, hospitalisation or quality of life, such as survival.

such as survival, stroke risk, hospitalisations or quality of life.

That raises a pretty fundamental question.

When an AI medical device is cleared for use, what exactly have we proved about it? The researchers also found that many of the studies that do exist were carried out in highly resourced healthcare systems and excluded groups including people over 75, pregnant women and non-English speakers.

And that matters beyond the US.

Countries around the world may rely on regulatory decisions made elsewhere, potentially making patients in lower-income countries inadvertent test populations for AI that hasn't been properly evaluated in their healthcare systems or communities.

So as AI becomes part of everyday medical care, should the rules change to make developers prove not just that their technology works, but that it actually makes a difference to patients? I spoke with Nura Issa from the Mubara University of Science and Technology in Uganda, Sebastian Ordonez, researcher in AI at MIT, and Leo Antonicelli, senior research scientist at MIT and associate professor at Harvard Medical School.

I started by asking Leo what does it mean when something is cleared by the FDA?