When AI meets medicine: Advancing multilingual care with GPT-4

One of GPT-4’s key strengths lies in its ability to extract explicit and straightforward information from medical notes. The model excelled in identifying patient demographics, medication details, and COVID-19 diagnoses, achieving sensitivity and specificity rates exceeding 90% for tasks such as recognizing patient age and obesity diagnoses. This capability makes GPT-4 a valuable tool for automating repetitive data entry tasks, reducing administrative burdens for healthcare professionals.

In the rapidly evolving field of healthcare technology, artificial intelligence (AI) promises to revolutionize how medical data is processed and utilized. A recent study titled “The Potential of Generative Pre-trained Transformer 4 (GPT-4) to Analyse Medical Notes in Three Different Languages: A Retrospective Model-Evaluation Study”, published in The Lancet Digital Health in January 2025, delves into GPT-4’s capabilities in analyzing medical notes across English, Spanish, and Italian. Conducted by a consortium of researchers from eight university hospitals, this study provides a nuanced understanding of GPT-4's strengths and limitations in clinical workflows.

A multilingual model for clinical note analysis

Medical notes are critical yet complex documents containing unstructured data that traditional systems struggle to interpret. The study evaluated GPT-4’s ability to process 56 de-identified medical notes from hospitals in the USA, Colombia, Singapore, and Italy. These notes encompassed admission, progress, and consultation notes written between February 2020 and June 2023. Each note was paired with 14 predefined questions designed to extract key medical insights, such as patient demographics, medical history, and diagnostic details.

GPT-4 demonstrated an overall accuracy rate of 79%, with notably higher agreement rates for Spanish (88%) and Italian (84%) notes compared to English (77%). The findings highlight the model’s potential for supporting clinical workflows while also revealing its limitations, particularly in tasks requiring inference and contextual understanding.

Strengths in explicit data extraction

One of GPT-4’s key strengths lies in its ability to extract explicit and straightforward information from medical notes. The model excelled in identifying patient demographics, medication details, and COVID-19 diagnoses, achieving sensitivity and specificity rates exceeding 90% for tasks such as recognizing patient age and obesity diagnoses. This capability makes GPT-4 a valuable tool for automating repetitive data entry tasks, reducing administrative burdens for healthcare professionals.

Additionally, GPT-4 performed exceptionally well in structured queries, such as identifying specific medical conditions or laboratory results explicitly mentioned in the notes. By automating these processes, the model could enhance the efficiency and accuracy of electronic health record (EHR) management, allowing clinicians to focus on patient care instead of administrative tasks.

Challenges with implicit information

While GPT-4 excelled at extracting explicit information, it struggled with tasks requiring inference or contextual reasoning. For example, when asked to identify whether multisystem inflammatory syndrome was a COVID-19 complication, the model often failed to make the connection. This limitation underscores the difficulty of applying general-purpose AI models to complex clinical scenarios where subtle patterns and relationships must be discerned.

Another challenge arose in identifying the primary reason for hospitalization, particularly in cases with multiple comorbidities. GPT-4’s reliance on textual cues often led to errors in prioritizing conditions, highlighting the need for domain-specific fine-tuning to improve its contextual understanding. These challenges limit the model’s applicability in critical decision-making processes, emphasizing the importance of human oversight.

Multilingual capabilities

A surprising outcome of the study was GPT-4’s higher accuracy in analyzing Spanish and Italian medical notes compared to English ones. Despite English dominating GPT-4’s training datasets, the simpler structure and shorter text length of non-English notes likely contributed to this result. This finding highlights the model’s adaptability to diverse linguistic contexts, suggesting its potential as a global tool for medical data analysis.

However, the study also underscores the importance of expanding training datasets to include more diverse languages and dialects. By doing so, GPT-4 could become a truly multilingual tool, capable of addressing healthcare challenges in underserved regions where non-English medical documentation is prevalent.

Implications for clinical practice

Enhancing Workflow Efficiency

GPT-4’s ability to automate data extraction and analysis could significantly reduce the administrative workload on clinicians. By summarizing patient notes, highlighting key insights, and streamlining documentation processes, the model can free up valuable time for healthcare providers to focus on patient care. This efficiency is particularly crucial in high-pressure environments such as emergency departments, where timely access to accurate information is critical.

Improving Patient Selection for Clinical Studies

The study demonstrated GPT-4’s effectiveness in identifying patients who meet inclusion criteria for hypothetical research studies. This capability could streamline clinical trial recruitment processes, making them faster and more cost-effective. By automating the initial screening process, GPT-4 could accelerate research timelines and improve access to cutting-edge treatments for eligible patients.

Limitations and Ethical Considerations

Despite its promise, GPT-4’s limitations in contextual understanding and inferential reasoning pose challenges for its integration into clinical decision-making. Additionally, the use of AI in analyzing sensitive medical data raises ethical concerns about data privacy, security, and accountability. Transparent algorithms and robust safeguards will be critical to building trust and ensuring responsible deployment in healthcare settings.

Recommendations for Future Research

Fine-Tuning for Clinical Tasks

Future research should focus on adapting GPT-4 and similar models to better handle complex medical inferences, such as identifying relationships between symptoms and underlying conditions. Domain-specific fine-tuning could enhance the model’s ability to support clinicians in making accurate and timely decisions.

Expanding Language Capabilities

Incorporating more diverse training datasets could improve GPT-4’s performance across additional languages and dialects. This expansion would make the model more inclusive, enabling it to address global healthcare challenges and bridge linguistic barriers in underserved regions.

Integration with Clinical Workflows

Research should explore how AI can complement existing healthcare technologies, focusing on seamless integration into EHR systems. Developing user-friendly interfaces and tools that align with clinical workflows will be essential for maximizing the model’s utility in real-world settings.

Ethical Frameworks

To address concerns about data privacy and security, researchers and policymakers must develop clear guidelines for handling medical data in AI-driven applications. These frameworks should prioritize transparency, accountability, and patient consent to ensure ethical deployment.