Bridging the gap: Transforming AI explanations into narratives everyone can understand

Explingo has the potential to revolutionize the way AI explanations are consumed across industries. In healthcare, for instance, natural-language narratives can help patients and clinicians better understand diagnostic predictions, fostering trust in AI-driven tools. Similarly, in finance, narrative explanations can clarify risk assessments for investors, enhancing transparency.

The growing reliance on machine learning (ML) in decision-making processes has highlighted the critical need for explanations of ML model predictions. Explanations for AI-driven predictions are essential for enabling informed decision-making across domains such as healthcare, finance, and education. However, traditional Explainable AI (XAI) outputs, often in the form of numerical values or complex visualizations, fail to bridge the gap between technical detail and human understanding. Prior studies indicate that users prefer explanations in a natural, narrative format, aligning with how humans naturally process information.

In their study “Explingo: Explaining AI Predictions using Large Language Models,” Alexandra Zytek and her co-authors from MIT and ESPACE-DEV IRD tackle this challenge. Submitted on arXiv, this research introduces Explingo, a system that leverages Large Language Models (LLMs) to transform ML explanations into intuitive, human-readable narratives. By combining LLM-generated narratives with automated grading systems, Explingo sets a benchmark for improving accessibility and usability in XAI.

The Explingo system: A dual approach

The Explingo system is divided into two subsystems: Narrator and Grader. Together, they form a pipeline for generating and evaluating natural-language narratives of AI predictions.

Narrator: Transforming explanations into narratives

The Narrator subsystem uses an LLM, such as GPT-4, to convert traditional ML explanations into narratives. This process involves a structured prompt that includes the context of the prediction, the format of the original explanation, and any exemplary narratives provided by users. For example, a SHAP explanation might detail how specific features of a house, such as square footage or location, contribute to its predicted price. The Narrator translates this into a narrative such as - "The 2,090 sq ft living space increases the predicted house price by approximately $16,400, while the additional second floor contributes another $16,200." This narrative style enhances user engagement and comprehension.

Grader: Evaluating narrative quality

Ensuring the quality of generated narratives is critical. The Grader subsystem uses an LLM to assess narratives across four metrics: accuracy, completeness, fluency, and conciseness. For example, the accuracy metric evaluates whether the narrative correctly represents the original explanation, while fluency assesses the naturalness of the language used. By automating the evaluation process, the Grader enables scalable, consistent quality checks across diverse applications.

Methodology and implementation

The Explingo system was developed through a rigorous, multi-stage process. Researchers created a series of exemplar datasets, including use cases from housing prices, student performance, and mushroom toxicity prediction. Each dataset included manually crafted narratives to serve as training examples for the Narrator subsystem. These exemplars were used to fine-tune the system's ability to generate domain-specific narratives while maintaining high-quality standards.

The Grader subsystem was validated against human-labeled datasets to ensure its scoring aligned with expert evaluations. The research team found that a combination of hand-written and bootstrapped exemplar narratives yielded the best balance between narrative fluency and technical correctness.

Explingo demonstrated significant advancements in making AI explanations more accessible. The system achieved high scores across all four evaluation metrics, particularly when guided by a small set of user-provided exemplar narratives. Notable findings include:

• Improved Comprehensibility: Narratives generated by Explingo were rated as highly understandable by users from non-technical backgrounds, bridging the gap between complex ML explanations and actionable insights.

• Versatility Across Domains: The system effectively adapted to diverse datasets, from predicting house prices to identifying toxic mushrooms, highlighting its generalizability.

• Scalable Evaluation: The automated Grader reduced the reliance on manual evaluation, enabling real-time quality assurance in live applications.

Practical implications

Explingo has the potential to revolutionize the way AI explanations are consumed across industries. In healthcare, for instance, natural-language narratives can help patients and clinicians better understand diagnostic predictions, fostering trust in AI-driven tools. Similarly, in finance, narrative explanations can clarify risk assessments for investors, enhancing transparency.

By making AI explanations more intuitive, Explingo can drive broader adoption of AI technologies, particularly in high-stakes domains where trust and understanding are paramount.

Challenges and the road ahead

Despite its promise, Explingo faces challenges that warrant further research. One limitation is its reliance on high-quality exemplar narratives, which may not always be available in new domains. Additionally, the system’s ability to handle complex explanations involving multiple interdependent features requires refinement.

Future developments could include incorporating additional metrics for narrative evaluation, such as emotional resonance or user engagement. Integrating interactive features, where users can ask follow-up questions to refine narratives further, would also enhance the system’s utility.