How can I explain my ML models to the business?
3 frameworks to make your AI more explainable
Explainability has been for sure one of the hottest topics in the area of AI — as there is more and more investment in the area of AI and the solutions are becoming increasingly effective, some businesses have found that they are not able to leverage AI at all! And why? Simple, a lot of these models are considered to be “black boxes” (you’ve probably already come across this term), which means that there is no way to explain the outcome of a certain algorithm, at least in terms that we are able to understand.
A lot of AI models are considered to be “black-boxes”
In the below picture we can see a complex mathematical expression with a lot of operations chained together. This image represents the ways the inner layers of a neural network function works. Seems too complex to be understandable, right?
What if I told you that the below expressions refer to the same neural network as the image above. Much easier to understand, right?
In a nutshell, this is the essence of Explainable AI — how can we translate the complex mathematical expressions involved in the process of training a black-box model in a way that businesses and persons understand it
This is called the Right for an Explanation, and it has definitely shaken how companies are implementing AI.
But with the needs and regulations coming out, there have been also new solutions targeting AI explainability popping up (yeyyy!!) to support businesses that want to leverage AI while being able to interpret them! No more black boxes, welcome to transparency! If you are curious to know more about why we need explainability, I suggest to check this article!
AI Explainability frameworks
This is a subject that has been explored by many authors — in 2016 in a seminar work by Marco Ribeiro, Sameer Singh, and Carlos Guestrin a novel solution for the interpretability of a black-box model was proposed. The proposed solution aimed at building two types of trust: trusting the prediction delivered by the model and trusting the model.
Since then many other frameworks and tools have been proposed to make AI explainability a reality across different data types and sectors. Today in this blogpost I’m covering — LIME, TF-Explain, and What-If.
LIME
Developed by researchers at the University of Washington to gain greater transparency on what happens inside of algorithm, LIME has become a very popular method within the community of Explainable AI.
When talking about developing a model on top of a dataset with low dimensionality, explainability might be easier, but when it comes to higher dimensions the complexity of the models also increases, which makes it very hard to maintain the local fidelity. LIME (Local Interpretable Model-Agnostic Explanations) tackles interpretability needs not only in the optimization of the models but also the notion of interpretable representation in a way that domain and task interpretability criteria are also incorporated.
There are a few examples of the combined use of LIME with common Data Science packages such as Scikit-Learn or XGBoost. Here you can check a practical example of AI explainability with Scikit-Learn and LIME.
ou can also take a deeper look at LIME’s tool, on their Github LIME.
Tf-explain your models!
Tf-explain is a library that was built to offer interpretability methods. Tf-explain implements interpretability methods while leveraging Tensorflow 2.0 callbacks to ease neural networks’ understanding. This useful package is offered to us by Sicara.
The library was built to offer a comprehensive list of interpretability methods, directly usable in your Tensorflow workflow:
- Tensorflow 2.0 compatibility
- Unified interface between methods
- Support for Training Integration (callbacks, Tensorboard)
The methods implemented in tf-explain are all already known from the literature like Activations Visualizations, Grad Cam, Occlusion Sensitivity, or Vanilla Gradients. All these flavors are meant for image explainability, but what about tabular data and time-series?
What-if?
What-if a framework with a cool and interactive visual interface existed, in order to better understand the output of TensorFlow models? The What-If tool is exactly that. Let’s say you need to analyze a previously deployed model, you can, regardless if it’s a model developed using Tensorflow or other packages such as XGBoost or Scikit-Learn.
Besides monitoring the models after deployment, you can also slice your datasets by features and compare performance across the different slices, while identifying in which subsets your models will perform better or worst. This not only helps with your model explainability but also opens the opportunity to research and understand topics such as bias and data fairness.
Here you can check an example of the What-If tool use in a Google Colab.
Final toughs
Explainability is without a doubt an important topic that will become one of the first concerns of companies in years to come, not only due to regulations but also because communities and persons are becoming more aware and sensitive to the potential of AI-based solutions.
Nevertheless and although significant and interesting progress has been made towards AI explainability in the last years, that are challenges still to be solved both on the methods and theory regarding the way the obtained explanations can be used in practice — for eg. explanations hat go beyond visualization and an objective evaluation of the quality of the explanations!
Explainability methods allow us o gain insights into how AI models work. But are all the answers we are looking for in the way models work? Or data quality and interpretability can play a massive role?
Fabiana Clemente is CDO at YData.
Improved and synthetic data for AI
