What's Up Doc? This AI Might Know Better Than Your Physician

(Bloomberg) -- Babylon Healthcare Services Ltd., the fast-growing mobile medical consultation service, said its artificial intelligence software, in tests, can assess common conditions more accurately than human doctors.

London-based Babylon said Wednesday its AI correctly answered 81 percent of diagnostic questions designed to mimic those trainee doctors must answer as part of the Royal College of General Practitioner's exam that must be passed to become a qualified GP doctor in the U.K. The exam is graded on a curve, but over the past five years, the average score trainees needed to pass was 72 percent.

But Martin Marshall, vice chairman of the Royal College of General Practitioners, which administers the exam Babylon's AI sought to benchmark itself against, said that software should never be compared to the abilities of human doctors.

“No app or algorithm will ever be able to do what a GP does,” Marshall said in a statement Wednesday. “Much of what a GP does is based on a trusting relationship between a patient and a doctor and research has shown that GPs have a 'gut feeling' when they just know something is wrong with a patient.”

It's not the first time Babylon has tangled with the RCGP either. The professional body has criticized the U.K. National Health Service for letting patients choose Babylon's "GP in Hand" mobile consultation service as their official primary care provider. It's also attacked Babylon for paying doctors who do consultations through its app too much money, making it more difficult for local NHS offices to find doctors to cover shifts.

Babylon demonstrated its AI technology publicly for the first time an event at London's Royal College of Physicians Wednesday, and said the company would publish the results of its tests online. It's made the tool available for free through its app and website in some parts of the U.K., including London, as well as Rwanda, although because the technology has not been approved by regulators, Babylon calls the software's answers "health information" not "diagnoses."

In addition to Babylon, others racing to create general diagnostic software include Ada, a startup with offices in Berlin and London that has launched a similar symptom-checker app to Babylon, HealthTap, in Palo Alto, California, Your.MD, a health information app created by British medical publisher BMJ, and International Business Machines Corp.

So far, AI companies' efforts to change health care have not always been smooth. IBM's Watson Health business has been criticized for falling short of promises that its system could help doctors select the best available cancer treatments for a given patient. Meanwhile, British regulators faulted a London hospital for transferring millions of patient records to Alphabet Inc.'s DeepMind.

Malcolm Grant, chair of NHS England, took part in a panel discussion during the launch event Babylon held to announce the AI test results Wednesday. He said Babylon's "GP at Hand" model was "interesting," but also sounded a note of caution. "The NHS has enormous intellectual capital and I want to be very clear about not driving that away," Grant said. "I want to be very clear that we will not drive GPs out of practice and see this as an Uber-style rival." 

There are also concerns about exactly how tools like Babylon's will be used. "There is already pushback from patients that they are not getting enough facetime with the doctor," said Michael Nash, a medical researcher and scientific advisor to Analytics Ventures. 

Ali Parsa, Babylon's founder and chief executive officer, took a different view. “There's no way to make health care accessible and affordable if we have to send people to doctors, and doctors take as long as they currently do with each patient," he said.

But the CEO agreed that in an ideal situation and in most developed countries, the best results would likely occur when doctors used the Babylon tool and in conjunction with their own skills and intuition. "They can improve each other," he said.

Parsa, a former physicist and investment banker, had previously founded Circle Holdings, a large partnership of private doctors that also sought to manage British hospitals on behalf of the government. It went public in 2011, but struggled to run hospitals profitably and was taken private by investment firm Toscafund in 2017.

Babylon said its existing service -- which lets people consult with a doctor over their mobile phone -- has 2 million users in Rwanda, a country where there are 0.064 doctors per 1,000 people, according to World Bank Data, compared to 2.8 in the U.K. and 2.3 in the U.S.

Babylon also pitted its software against a group of seven experienced primary care doctors on a set of 100 hypothetical cases developed by primary care experts at the Royal College of Physicians and the health systems at Stanford University and Yale University. Babylon's software made the correct diagnosis in 80 percent of cases, while the human docs' accuracy ranged from 64 percent to 94 percent, it said.

When the software was presented with a smaller set of cases that represented the conditions most commonly seen by GPs, its accuracy was 98 percent, the company said.

To contact the editor responsible for this story: Nate Lanxon at nlanxon@bloomberg.net, Giles Turner

©2018 Bloomberg L.P.