Medicine is one of the hottest fields when it comes to applying AI to real-world problems, in particular using deep learning systems to detect disease in medical imagery.
Now the authors of a paper in The Lancet Digital Health have carried out the first systematic review and meta-analysis of all studies between January 2012 and June 2019 comparing deep learning models’ ability to detect disease through medical imaging to that of health professionals.
The trawl found 20,500 articles tackling the topic, but shockingly, less than 1 percent of them were scientifically robust enough to be confident in their claims, say the authors. Of those, only 25 tested their deep learning models on unseen data, and only 14 actually compared performance with health professionals on the same test sample.
Nonetheless, when the researchers pooled the data from the 14 most rigorous studies, they found the deep learning systems correctly detected disease in 87 percent of cases, compared to 86 percent for healthcare professionals.
Ultimately, then, the results of the review are broadly positive for AI, but damning of the hype that has built up around the technology and the research practices of most of those trying to apply it to medical diagnosis.
Read full, original post: AI Can Diagnose Like Doctors. But for Continued Progress, Research Standards Must Improve