Safeguarding Students: Effective AI Integration in WhatsApp-Based Intelligent Tutoring Systems

The study explores integrating large language models into Intelligent Tutoring Systems to enhance personalized learning while addressing safety risks, finding effective strategies to moderate inappropriate content and ensure safe student interactions. Researchers implemented a WhatsApp-based chatbot for math tutoring in African countries, demonstrating the importance of continuous monitoring and ethical deployment in educational technologies.

A study by Zachary Levonian of the Digital Harbor Foundation in Baltimore, Maryland, and Owen Henkel of the University of Oxford, explores the integration of large language models (LLMs) into Intelligent Tutoring Systems (ITSs) to enhance personalized learning while addressing safety concerns. Researchers developed a conversational system for a WhatsApp-based chatbot, Rori, which tutors math to middle-school students in Sierra Leone, Liberia, Ghana, and Rwanda. The system aims to teach growth mindset concepts and support math practice while ensuring student safety through careful design and moderation.

Addressing Safety Risks in LLM-Based Education

The study highlights three primary safety risks: the generation of harmful content by the system, endorsement of harmful content, and the provision of incorrect or harmful advice. To mitigate these risks, the researchers implemented a safety filter using a word list and OpenAI's moderation API. The system redirects or ends conversations based on the assessed risk of student messages. Empirical data from over 8,000 student interactions shows that GPT-3.5 rarely generates inappropriate messages, with most inappropriate content coming from students. This underscores the importance of content moderation and classroom management. The researchers conducted two studies: an in-classroom usability test and a field deployment. The results indicate that the semi-structured conversation design effectively prevents harmful content generation and addresses inappropriate student messages.

Classroom Usability Test: Promising Results

In the first study, 109 in-school students across six classrooms used the growth mindset generative chat during a regularly-scheduled study hall. The study involved 252 conversations, with a 60% completion rate. Students were asked to rate the conversations, and 16 out of 252 conversations were rated less than five stars. A qualitative investigation of these low-rated conversations revealed no significant difference from five-star conversations. No student or GPT-3.5 messages were flagged by the safety filter, indicating the effectiveness of the semi-structured conversation design in preventing inappropriate content. The highest moderation score for a GPT-3.5 message was 0.01, while the highest student message score was 0.05, which was a typo.

Field Deployment: Broadening the Scope

The second study, a field deployment, analyzed 126,278 messages from the feature launch to May 1, 2024. The growth mindset conversation was deployed for non-school users of Rori and incorporated into the onboarding process before math skills practice. The study found that 0.31% of student messages received a score of at least 0.1 in any moderation category, with fewer than 8 in 10,000 messages flagged. The most common negative messages were harassing or sexual, but only one message was flagged as high risk and determined to be a false positive. The researchers investigated messages generated in response to student messages near the safety filter thresholds. They found that 40 out of 48 unflagged conversations contained student messages that warranted caution, and GPT-3.5 responses were appropriately corrective in 37 cases. In three cases, the responses ignored or equivocated when a corrective message was warranted, a subtle form of the yea-sayer effect.

Challenges and Future Directions

The paper discusses the challenges of handling inappropriate or sensitive student inputs, drawing parallels with content moderation on online platforms. It emphasizes the need for ongoing safety monitoring and culturally responsive moderation strategies. The researchers highlight the importance of red-teaming exercises in identifying potential risks and building organizational confidence. They found that transparency about the system's shortcomings and involving designers, educators, and researchers in the evaluation process improved trust and solicited higher-quality feedback.

Ensuring Ethical and Safe Learning Environments

The study also found that the specific moderation actions implemented were reasonable starting points. By classifying messages at two risk levels, the system positively redirected conversations with pre-vetted messages. Corrective messages were written by educators, and future work aims to combine culturally-responsive classroom management with soliciting cultural background information from students. In serious disclosures, the researchers argue that automatically ending the conversation and moving to human review is more ethical than generating an LLM response at the moment. However, they suggest technical infrastructure that starts an in-chat support session with a human or explicitly connects to contacts at the student’s school.

The paper concludes by noting the importance of continual monitoring for yea-sayer effects and developing technical approaches that explicitly model appropriate corrective behavior. The study provides insights into the design and evaluation of safe generative chats in educational settings, emphasizing the need for continuous improvement in content moderation and ethical deployment of LLMs in ITSs to ensure positive student experiences. This work demonstrates the potential of LLMs to enhance educational outcomes while highlighting the critical importance of safeguarding measures to protect students in digital learning environments.