The Massachusetts Institute of Technology (MIT) is bridging the gap between esoteric computational predictions and everyday understanding. Researchers at MIT have developed a two-tiered system called EXPLINGO, which translates complex AI model explanations into plain language. This could be a game-changer for users who need to evaluate the trustworthiness of AI without mastering machine-learning themselves.
At the core of this innovation is the handling of SHAP explanations, which could be cumbersome due to their intricate visualizations and technical depth. A SHAP explanation assigns values to the data features that an AI model uses to make a prediction. These values determine how much each feature influences the final outcome. Despite SHAP's potential for clarity, the original bar plot presentation quickly becomes unwieldy when dealing with models that analyze hundreds of features, according to a statement obtained by MIT News.
To simplify this complexity, the MIT team's system leverages large language models (LLMs) to generate narratives that describe SHAP explanations in language that is accessible to the average user. Alexandra Zytek, an electrical engineering and computer science (EECS) graduate student at MIT and lead author of the study, highlighted their approach. "Our goal with this research was to take the first step toward allowing users to have full-blown conversations with machine-learning models about the reasons they made certain predictions," Zytek told MIT News.
EXPLINGO consists of two components: NARRATOR, which generates the narrative from the SHAP explanation, and GRADER, which then evaluates that narrative. The evaluation is based on conciseness, accuracy, completeness, and fluency, allowing users to weigh these metrics according to the specific needs of their scenarios. For instance, in critical situations, accuracy and completeness might be prioritized over fluency in the narrative's assessment.
Meticulous testing went into refining this system. The researchers challenged EXPLINGO by having different users pen narratives for multiple datasets, assessing the system's malleability and the quality of its output. The results were promising, indicative of EXPLINGO's proficiency in mimicking various narrative styles and generating high-quality explanations. The research, set to be presented at the IEEE Big Data Conference, points toward a future where discussions with AI about its decision-making processes are more intuitive and user-friendly. "If people disagree with a model’s prediction, we want them to be able to quickly figure out if their intuition is correct, or if the model’s intuition is correct, and where that difference is coming from," Zytek explained to MIT News.
