EHRmonize: Revolutionizing EHR Data Abstraction with AI, Reducing Annotation Time by 60 percent

EHRmonize leverages large language models to automate and enhance the abstraction of medical concepts from electronic health records, significantly reducing annotation time by 60% and improving data harmonization for healthcare research. This framework, which outperformed others in various tasks, underscores the importance of clinician oversight and is now available as an open-source Python package.

Electronic health records (EHRs) contain vast amounts of complex data, but harmonizing and processing this information remains a challenging and costly task that requires significant clinical expertise. While large language models (LLMs) have shown promise in various healthcare applications, their potential for abstracting medical concepts from EHRs has not been fully explored. Addressing this gap, Joao Matos, Jack Gallifant, Jian Pei, and A. Ian Wong from Duke University and the Massachusetts Institute of Technology introduced EHRmonize, a framework leveraging LLMs to abstract medical concepts from EHR data. EHRmonize was tested using medication data from two real-world EHR databases to evaluate five LLMs on two free-text extraction and six binary classification tasks. The study found that GPT-4o, with 10-shot prompting, achieved the highest performance in all tasks, identifying generic route names with 97% accuracy, generic drug names with 82% accuracy, and performing binary classification of antibiotics with 100% accuracy. This framework significantly enhances efficiency, reducing annotation time by approximately 60%, though it emphasizes that clinician oversight remains essential.

Revolutionizing EHR Data Processing with LLMs

The variability in recording practices, inconsistent terminology, local coding systems, and evolving standards pose significant challenges in leveraging EHR data. These challenges often lead to time-consuming, labor-intensive, and error-prone manual abstraction processes, limiting the accessibility of EHR data for many researchers. EHRmonize automates these processes, potentially accelerating healthcare research and improving data harmonization. The framework includes two components: corpus generation, which involves SQL-based extraction of relevant text and concepts from EHR databases, and LLM inference, which converts raw input into standardized classes via few-shot prompting. The study used five state-of-the-art LLMs, including Anthropic’s Claude-3.5-Sonnet, Meta’s Llama3-70B, Mistral’s Mixtral-8x7B, and OpenAI’s GPT-3.5-Turbo and GPT-4o. Among these, GPT-4o consistently outperformed the others, demonstrating the potential of LLMs in abstracting medical concepts from structured EHR data. Extensive evaluations across various prompting strategies revealed that GPT-4o and Claude-3.5-Sonnet maintained high performance with increasing examples, while GPT-3.5-Turbo’s performance declined as the number of examples increased. Variations in temperature up to 0.5 had minimal impact on model performance across tasks. EHRmonize demonstrated significant efficiency gains, reducing annotation time by approximately 60%.

EHRmonize: Enhancing Efficiency and Reducing Errors

Despite its promising results, the study acknowledged several limitations, including the limited size and scope of the dataset, which focused solely on medication data. Future research should expand the dataset’s volume and scope, incorporating other domains such as laboratory results and flowsheet data. Additionally, exploring the impact of example ordering in few-shot prompting and incorporating semantic equivalence in free-text evaluation could provide valuable insights into prompt engineering for medical NLP tasks. EHRmonize, now available as a Python package on PyPI, represents a significant step towards lowering barriers in EHR data research. By improving abstraction efficiency for structured data fields—a task often performed manually EHRmonize has the potential to accelerate research and enable more comprehensive analyses of EHR data. This framework demonstrates the potential of LLMs to enhance, rather than replace, manual chart review by prepopulating options and allowing clinicians to focus on more complex abstraction tasks.

Automating Medical Concept Abstraction for Better Research

EHRmonize offers a scalable and adaptable solution to the challenges of EHR data harmonization. Its open-source implementation aims to foster collaboration and accelerate progress in the field of EHR data science, potentially transforming how researchers and clinicians interact with EHR data. EHRmonize’s development was driven by the recognition that manual methods of EHR data abstraction are not only time-consuming but also prone to errors and heavily dependent on clinical expertise. The traditional approaches to data harmonization and abstraction often involve manual cleaning, categorization, and summarization of concepts, which are labor-intensive and limit the accessibility of EHR data for many researchers. By automating these processes, EHRmonize can significantly reduce the time and effort required for data preparation, thus enabling more researchers to leverage EHR data in their studies.

Bridging the Gap in Medical Data Harmonization

EHRmonize addresses the critical need for efficient, scalable methods to harmonize EHR data, which is often recorded inconsistently across different institutions and even within the same hospital. This inconsistency, coupled with the use of local coding systems and evolving medical terminologies, poses significant challenges to data processing and harmonization. By leveraging the capabilities of LLMs, EHRmonize can automate the categorization and harmonization of EHR entries, addressing many of these challenges and lowering barriers to entry for researchers. The framework's ability to significantly enhance efficiency by reducing annotation time by approximately 60% highlights its potential to transform healthcare research and data analytics. While clinician oversight remains essential to ensure accuracy and reliability, EHRmonize provides a powerful tool that can assist clinicians and researchers in EHR data abstraction, ultimately contributing to more efficient and comprehensive healthcare research.