In a study of breast biopsies, diagnostic agreement — the percentage of doctors who settled on a single diagnosis — was as low as 48 percent. And a 1968 study found that physicians, when given a copy of a stomach ulcer screening they’d already diagnosed, tended to disagree with themselves, rendering entirely different diagnoses.
It’s no real wonder, then, that algorithms are discovering insights that human doctors can’t. A recent Google project focused on analyzing breast biopsies used data from Camelyon16 project, which challenged participants to create cancer-detection algorithms. With the accumulated data, Google has developed a model that can match or exceed the performance of a pathologist taking an unlimited amount of time to examine images.
In preliminary tests, Google’s algorithm not only performed better than a human pathologist, but did just as well in a generalized scenario with images acquired from a different hospital. And in at least two instances, it did so faster than its human competition.
That’s not entirely surprising. As Google researchers point out in a recent blog post, pathologists are responsible for reviewing all biological tissue on a slide, and there are typically many slides per patient. Making matters worse, each slide is 10+ gigapixels when digitized at 40X magnification — a veritable mountain of information to sift through in a few hours. “Imagine having to go through a thousand 10-megapixel (MP) photos, and having to be responsible for every pixel,” Google’s Martin Stumpe, and Lily Peng wrote.
Google’s research is good news for cancer patients, for whom the diagnostic stakes are especially high. Diagnoses that are missed, incorrect, or delayed are believed to affect 10 to 20 percent of cases, and a 2009 report funded by the federal Agency for Healthcare Research and Quality found that 28 percent of 583 diagnostic mistakes were life-threatening or resulted in death or permanent disability.
Google notes that the algorithm isn’t a silver bullet — it can’t account for an experienced pathologist’s breadth of knowledge, for example. That’s why the search giant envisions them incorporated “in a way that complements the pathologist’s workflow,” rather than replacing the human element altogether. “We envision that algorithms such as ours could improve the efficiency and consistency of pathologists,” Stumpe wrote. “For example, pathologists could reduce their false negative rates (percentage of undetected tumors) by reviewing the top ranked predictive tumor regions including up to 8 false positive regions per slide.”
