Artificial intelligence has proven itself useful in reading medical imaging and even shown it can pass doctors’ licensing exams.
Now, a new AI tool has demonstrated the ability to read physicians’ notes and accurately anticipate patients’ risk of death, readmission to hospital, and other outcomes important to their care.
Designed by a team at NYU Grossman School of Medicine, the software is currently in use at the university’s affiliated hospitals throughout New York, with the hope that it will become a standard part of health care.
A study on its predictive value was published Wednesday in the journal Nature.
Lead author Eric Oermann, an NYU neurosurgeon and computer scientist, told AFP that while non-AI predictive models have been around in medicine for a long time, they were hardly used in practice because the data they needed requires cumbersome reorganization and formatting.
But “one thing that’s common in medicine everywhere, is physicians write notes about what they’ve seen in clinic, what they’ve discussed with patients,” he said.
“So our basic insight was, can we start with medical notes as our source of data, and then build predictive models on top of it?”
The large language model, called NYUTron, was trained on millions of clinical notes from the health records of 387,000 people who received care within NYU Langone hospitals between January 2011 and May 2020.
These included any records written by doctors, such as patient progress notes, radiology reports and discharge instructions, resulting in a 4.1-billion-word corpus.
One of the key challenges for the software was interpreting the natural language that physicians write in, which varies greatly among individuals, including in the abbreviations they choose.
By looking back at records of what happened, researchers were able to calculate how often the software’s predictions turned out to be accurate.
They also tested the tool in live environments, training it on the records from, for example, a hospital in Manhattan then seeing how it fared in a Brooklyn hospital, with different patient demographics.
Not a substitute for humans
Overall, NYUTron identified an unnerving 95 percent of people who died in hospital before they were discharged, and 80 percent of patients who would be readmitted within 30 days.
It outperformed most doctors on its predictions, as well as the non-AI computer models used today.
But, to the team’s surprise, “the most senior physician who’s actually a very famous physician, he had superhuman performance, better than the model,” said Oermann.
“The sweet spot for technology and medicine isn’t that it’s going to always deliver necessarily superhuman results, but it’s going to really bring up that baseline.”
NYUTron also correctly estimated 79 percent of patients’ actual length of stay, 87 percent of cases where patients were denied coverage by insurance, and 89 percent of cases where a patient’s primary disease was accompanied by additional conditions.
AI will never be a substitute for the physician-patient relationship, said Oermann. Rather, they will help “provide more information for physicians seamlessly at the point-of-care so they can make more informed decisions.”
More information:
Eric Karl Oermann, Health system scale language models are all-purpose prediction engines, Nature (2023). DOI: 10.1038/s41586-023-06160-y. www.nature.com/articles/s41586-023-06160-y
2023 AFP
Citation:
‘AI doctor’ better at predicting patient outcomes, including death (2023, June 7)
