Using artificial intelligence (AI) to analyze clinical reports in electronic health records can accurately identify patients who have implantable devices that pose a safety risk for MRI exams, according to research published online August 12 in the Journal of the American College of Radiology.
A Harvard Medical School team led by Vladimir Valtchinov, PhD, compared two different approaches of natural language processing (NLP) techniques: an "expert"-derived method preloaded with relevant search terms and an ontology-driven technique that extracts medical terms based on publicly available biomedical ontologies.
Using radiology reports, emergency department notes, and other clinical notes, the group found that both AI-based approaches yielded high accuracy for identifying patients with implantable devices who are a high safety risk for MRI.
For the expert-driven method, the researchers used a search algorithm from Queriable Patient Inference Dossier (QPID), a software platform being developed by a company founded by researchers from the Massachusetts General Hospital Research Institute. Utilizing published criteria for MRI safety labeling, the QPID algorithm employs a curated list of single terms or phrases to separately identity devices with higher safety risk -- and that would have absolute contraindications for MRI -- as well as devices at a lower safety risk, which are considered to have relative contraindications.
The ontology-driven NLP system was a customized version of the open-source clinical Text Analysis and Knowledge Extraction System (cTAKES), which had been mapped with clinical terms from the Systematized Nomenclature of Medicine -- Clinical Terms (SNOMED-CT) vocabulary files and process definitions, as well as radiology terms from RadLex.
They first used both approaches to query 174,769 emergency department notes, 25,000 radiology reports, and 41,085 other clinical reports. The researchers then compared the methods with manual review performed on a random sample of 465 reports, including 155 samples from each of the report categories. The ontology-driven NLP method and the expert-derived NLP technique yielded comparable performances.
| Performance of AI for identifying implantable devices with high MRI safety risk | ||
| Ontology-driven NLP method | Expert-derived NLP method | |
| Sensitivity | 82% | 88% |
| Specificity | 90% | 96% |
| Accuracy | 83% | 91% |
The methods also posted statistically similar results for finding patients with lower safety risk implantable devices, with one exception: The ontology-driven NLP method had slightly higher specificity (62% versus 57%, p = 0.036) for identifying these devices.
"Our study supports the notion that ontology-driven NLP may be a suitable and effective alternative for expert-driven NLP in targeted clinical data retrieval tasks," the authors wrote.
In other findings, the researchers noted that both approaches retrieved a greater proportion of reports with implantable devices from radiology reports compared with other data types. Future studies exploring the use of automated methods for retrieving reports with implantable devices that may pose an MRI safety risk might be appropriate, they concluded.
"In a clinical workflow when an MR imaging is ordered, a trigger to analyze clinical records before the examination would be helpful to inform providers and patients about potential safety risks," the authors wrote. "This step could be added either before patient arrival or right before an examination."
