New frontiers in Explainable AI
AI is astonishing: it can drive cars, answer questions, match people’s faces with their passport photos, beat the best champions at chess, and much more… But, have you ever wondered how it works? And what happens when it makes mistakes? Will it ever become dangerous?
We are far from Terminator-like catastrophic events, but the problem is real. AI now competes and sometimes outperforms people on many tasks thanks to the development of new learning algorithms, particularly neural networks. These algorithms can discover patterns in large datasets and adapt their answers when new data is provided without any human interventions. However, they are based on big conglomerates of non-linear mathematical functions. The resulting models are so convoluted that even researchers struggle to understand how they can reach such astonishing performances, never mind justifying wrong predictions or unexpected behaviours. This can hinder further developments of AI as the model’s opaqueness impedes the researcher from finding the weak spots. Plus, the widespread use of AI is urging to make this technology transparent as erroneous predictions can already dramatically impact people’s lives. The European Parliament ruled out that AI systems fall within the General Data Protection Regulation scope and must explain their fully automated predictions involving the usage and analysis of sensitive data.
So, researchers created a new branch of AI, called eXplainable AI (XAI), to solve this problem. They have developed a plethora of XAI methods to expose and explain how learning algorithms work, so people can comprehend and trust their results. A torrent of scientific studies in XAI has been published every year since the mid-2010s. Despite the massive number of scientific publications, there are still critical gaps in the XAI fields.
Learning algorithms are trained over various learning algorithms and data types, ranging from numbers to texts and including images and videos. It is unlikely that a single XAI method can produce effective and meaningful explanations for every application, thus the flourishing stream of novel XAI solutions. It has become necessary to organise these methods within a conceptual framework, which can guide AI practitioners to choose the proper XAI method for the problem at hand. Over time, many classification systems have been proposed, and there is now a general consensus on organising the XAI methods according to the following dimensions:
- The scope of an explanation. A global explanation attempts to make the model transparent and understandable as a whole. A local explanation focuses on explaining each prediction of a model.
- The stage at which an XAI method generates explanations. Ante hoc methods modify a model before training to make it naturally understandable whilst still achieving optimal accuracy. Post hoc methods keep a trained model unchanged and use an external explainer to understand its functioning.
- The problem type is to be solved by AI, including classification, regression or clustering.
- The input data (numerical/categorical, pictorial, textual or times series) can play an essential role in constructing a model and, consequently, its XAI method.
I have recently proposed adding a fifth dimension that considers the explanation format produced by an XAI method [1]. Similarly to input data, different applications require different explanations. The formats that have been tested until now are numerical, rules, textual, visual or a mix of the previous four (see Figure 1).
Researchers have applied these explanations in various contexts with mixed results. It is difficult to say which explanation format is the most effective and understandable. There is no general consensus on when an explanation is better than another one. Scholars have identified numerous concepts that can affect the quality of an explanation. I put together a list of these concepts [2] as the first step of selecting the most relevant ones to generate a meaningful explanation for humans. As per the XAI methods, it is unlikely that the scientific community will develop a one-fits-all solution as people think and reason in different ways. However, determining which explanation formats work better in certain circumstances will represent a huge step forward towards making AI transparent and user-friendly. There is also the need to carry out more studies with human participants to collect their opinions on the various explanations and understand the most comprehensible ones. Researchers are working hard to close all these gaps in XAI and make AI more friendly for people. I am sure that we will witness very interesting breakthroughs in the near future. However, it could be even more interesting to get practitioners and final users involved in the discussion as their feedback can help researchers find the best solutions. Explainability is such a wide topic that requires the contribution of the largest possible community.
References
[1] Vilone, G., and Longo, L., Classification of explainable artificial intelligence methods through their output formats (2021), Machine Learning and Knowledge Extraction, 3(3), 615–661.
[2] Vilone, G., and Longo, L., Notions of explainability and evaluation approaches for explainable artificial intelligence (2021), Information Fusion, 76, 89–106.
