The utility of imaging clinical decision-support (CDS) software can be enhanced by integrating artificial intelligence (AI), according to research presented May 24 at the Society for Imaging Informatics in Medicine (SIIM) annual meeting.
After implementing AI as part of their CDS installation, researchers from Yale School of Medicine found that clinicians were more likely to provide the software with a structured clinical indication for ordering advanced imaging studies. That led to a significant improvement in appropriate orders.
"Free-text entry of exam indication with AI-assisted structured indication selection improved CDS scoring and advanced imaging order appropriateness," said presenter Dr. Dorothy Sippo.
When their health system implemented clinical decision support in 2019 for outpatient radiology orders, ordering providers initially had difficulties in identifying the appropriate structured indications required by the CDS software, according to Sippo. In 46% of orders, the software couldn't score the appropriateness of the exam because the ordering clinician did not select a structured indication.
"This was also challenging for radiologists, because they were not consistently receiving meaningful and complete clinical history," Sippo said. "So from both the CDS functioning standpoint as well as the radiologists getting the information that we're needing, we were facing some challenges with our CDS implementation."
In September 2020, the institution upgraded its commercially available CDS software to enable ordering physicians to enter exam indication as free text instead of having to search for a checkbox with a structured indication, she said. After the AI software suggests a structured indication based on the free text, the provider then selects the structured indication from the AI's suggestion -- either via an automatic prediction from the software or by choosing from a list of indications provided in a pop-up on the software. Radiologists then receive both the free-text and structured indication information once the order is complete.
The group publicized the new AI functionality with a presentation to the health system leadership, an informational email, and an instructional video. However, it was challenging to inform all providers of the new AI functionality, she said. Some providers were also still finding ways to bypass CDS, such as closing the pop-up with the AI-suggested structured indications. As a further complication, other updates were added to the CDS software -- including content changes for oncologists -- that caused other difficulties, Sippo said.
The integration of AI did, however, deliver a significant improvement in the number of appropriate exam orders. And the AI functionality was used in 38% of orders, she said.
"We were also excited to see that there was a decrease in the number of our unscored orders, which is exactly what we wanted," she said. "This is why we implemented the AI functionality."
| Impact of AI on CDS ordering results | |||
| Q1 2020 (before AI) | Q2 2020 (before AI) | Q4 2020 (after AI) | |
| Appropriate orders | 41% | 37% | 50% |
| Marginal orders | 12% | 9% | 13% |
| Inappropriate orders | 5% | 4% | 6% |
| Unscored orders | 42% | 50% | 31% |
The researchers learned a few lessons along the way, including that providers can enter specific detailed clinical history in a free-text format. This enabled radiologists to receive more granular data, she said.
In addition, the AI capability enabled structured information to be collected from these free-text clinical histories, Sippo said. What's more, the authors learned that it's essential to communicate clearly and broadly to all providers in order for new electronic health record functionality to be successfully adopted, Sippo said.
"When we saw that providers were bypassing the CDS workflow, we realized they weren't clear on how to use it and we had to engage them further," she said. "I think there is still work to be done for us in this area."
On the downside, Sippo noted that even with the assistance of AI, 31% of orders were still unscored by the CDS software. The researchers believe there's room for improvement, however.
"Because we know that unless there is a very close match between the free text entered and the AI's predicted structured indication, the provider must select from a pop-up list of predicted indications," Sippo said.
Some of the providers are choosing to bypass the CDS by closing this pop up, according to Sippo. In the next month, the group plans to test a new workflow that will lower the threshold for the AI to automatically select a predicted structured indication based on its analysis of the provider's free-text indication.
