The world’s leading publication for data science, AI, and ML professionals.

SHAP: Shapley Additive Explanations

A step-by-step guide for understanding how SHAP works and how to interpret ML models by using the SHAP library

Figure 1. SHAP | Image by author | Icons by freepick
Figure 1. SHAP | Image by author | Icons by freepick

In the past decade, we have witnessed the explosion of the age of artificial intelligence, driven by academia and embraced by the industry. Such has been the inclusion of AI in everyday life that even business models revolve around artificial intelligence models.

AI has shown impressive results in areas such as natural language processing, computer vision, etc. Today, it takes just a few lines of code to implement state-of-the-art AI models, it’s fascinating. However, as humans, how can we interpret the predictions that AI models make? How can we measure the importance that the model gives the data to be inferred? Well, in this blog we will talk about this controversial topic: the interpretability of artificial intelligence models.

The Interpretability of AI models is an active research area. Several alternatives have been proposed in recent years, and in this blog, we will focus on one in particular: SHAP (Shapley Additive Explanations).

In this blog we are going to see the interpretability of ML models from the SHAP approach, we will see how SHAP works and its main component: Shapley Values. Therefore, this blog will be divided into the following sections:

  • What are Shapley Values?
  • What is SHAP?
  • SHAP in action: A classification problem
  • Conclusion

I recommend you go for a cup of coffee and make yourself comfortable ☕️!

What are Shapely values?

Shapley values are a concept of the cooperative game theory field, whose objective is to measure each player’s contribution to the game. The method for obtaining Shapley values was proposed by Lloyd Shapley [1] in 1953. Shapley values emerge from the context where "n" players participate collectively obtaining a reward "p" which is intended to be fairly distributed at each one of the "n" players according to the individual contribution, such a contribution is a Shapley value.

In simple words, a Shapley value is the average marginal contribution of an instance of a feature among all possible coalitions. Average marginal contribution? all possible coalitions? Let’s see in detail what all this refers to.

Let’s say that a group of friends (A , B , C , D ) is working together obtain a profit P . To distribute the profit fairly, it is intended to measure the contribution of each member, that is, the Shapley value of every friend. To calculate the Shapley value of a given member, the difference between the profit that is generated when the member is present is calculated with respect to the profit that is generated when that member is absent (such difference is the marginal contribution of the given member to the current coalition), this is done for each all the subgroups (or coalitions) that can be generated where the member for whom you are looking to calculate your contribution is present. The mean of the differences obtained (the mean marginal contribution) is the Shapley value.

In the following figure, we see a representation of the calculation of the marginal contribution of the friend A to the collation composed of friends B , C and D.

Figure 2. Marginal Contribution of member "A" to the coalition of members B, C, D. | Image by author | Icons by freepick
Figure 2. Marginal Contribution of member "A" to the coalition of members B, C, D. | Image by author | Icons by freepick

For example, to calculate the Shapley value of the friend A we would need to construct all the collations where the friend A appears, for each collation, the marginal contribution would be calculated (the difference between the profit obtained when the member is present versus the profit obtained when the member is absent) and given all marginal contributions, the mean marginal contribution would be calculated, that is, the Shapley value. Simple, right?

Figure 3 shows the representation of the calculation of the Shapley value for the friend A.

Figure 3. Shapely value calculation for member "A" | Image by author | Icons by freepick
Figure 3. Shapely value calculation for member "A" | Image by author | Icons by freepick

Well, up to this point we already know what Shapley value is, how it is calculated and what is its interpretation. However, how are Shapley values related to the interpretability of ML models? Let’s see how this happens.

In the context of an ML model, let’s think that each friend in our example is a feature, the game is the function that generates predictions, and the profit is the prediction.

However, ML models usually have a large number of features where each feature is a discrete or continuous variable, this causes it to be computationally very complicated to calculate Shapley values for each instance of each feature, in fact, it is an NP-hard problem.

And it is at this point where SHAP becomes the protagonist. In the next section, we will see what SHAP is and its approach to the interpretability of ML models.

Maybe it’s time to refill the coffee cup ☕️!

What is SHAP?

Shapley Additive Explanations (SHAP), is a method introduced by Lundberg and Lee in 2017 [2] for the interpretation of predictions of ML models through Shapely values. The key idea of SHAP is to calculate the Shapley values for each feature of the sample to be interpreted, where each Shapley value represents the impact that the feature to which it is associated, generates in the prediction.

The intuition behind SHAP is easy to understand, for each feature there is an associated Shapley value. However, how does SHAP work? In the previous section, we saw that the calculation of Shapley values can become intractable for many features. Well, to avoid this, the authors introduced Kernel Shap, an extended and adapted method from linear LIME [3] to calculate Shapley values.

Kernel Shap is a method that allows the calculation of Shapley values with much fewer coalition samples. Kernel Shap is based on a weighted linear regression where the coefficients of the solution are the Shapley values. To build the weighted linear model, n sample coalitions are taken, for each coalition, the prediction is obtained and the weight is calculated with the kernel shap. Finally, the weighted linear model is fit and the resulting coefficients are the Shapley values.

A bit complicated, right? Let’s see it in detail.

Let’s say we want to explain the instance x , which consists of features f1 , f2 , f3 , and f4 . The procedure begins by taking a set of coalition samples. For example, the coalition (1,0,1,0) refers to the presence of the features f1 and f3 , and the absence of the features f2 and f4 . Since ML models cannot omit features to make inferences, the values of features f2 and f4 are replaced by values taken from the training set. Then, for the coalition (1,0,1,0)the values of features f1 and f3 are taken from the instance x and the values of features f2 and f4 come from the training set, in this way the prediction can be made correctly. Therefore, for each coalition sample, the prediction is obtained and the weight is calculated with the kernel shap described in equation 1. Solving the linear model, the resulting coefficients are the Shapley values.

Equation 1. Kernel Shap
Equation 1. Kernel Shap

Everything is clearer, right? Nothing is better than an image that illustrates the entire process. In Figure 4 we see the descriptive process for the calculation of Shapley values for a given instance X.

Figure 4. Descriptive process of SHAP to obtain Shapley values from an ML model | Image by author
Figure 4. Descriptive process of SHAP to obtain Shapley values from an ML model | Image by author

As we can see from the figure above, the process to obtain Shapley values from an ML model through Kernel Shap is not a complicated process but it is laborious. The main ingredients are coalitions, predictions, and weights.

It is important to note that trying to calculate all the coalitions makes the problem intractable. This is the reason why coalition samples are taken, the larger the sample, the lower the uncertainty.

Finally, it is important to say Kernel Shap is the only method model-agnostic for the calculation of Shapley values, that is, Kernel Shap can interpret any ML model regardless of its nature. On the other hand, the authors also propose other variants to obtain Shapley values based on different types of models such as Tree SHAP, Deep SHAP, Low-Order SHAP, Linear SHAP, and Max SHAP.

Well, it’s time to see SHAP in action. In the next section, we will see how to use the shap library to obtain Shapley values from an ML model. I think it’s time to refill the coffee cup ☕️!

SHAP in action: A classification problem

SHAP and its variants are integrated into the python library shap , which, in addition to providing different methods for calculating Shapely values, also integrates several methods for the visualization and interpretation of results.

The goal of this section is to show how to use the shap library to calculate, plot, and interpret Shapley values from a classification problem. So, let’s go for it!

The complete project, with the examples shown in this article, can be found at: https://github.com/FernandoLpz/SHAP-Classification-Regression

The dataset

For this example, we will use the Breast Cancer Wisconsin (Diagnostic) dataset [4], whose features describe the characteristics of a tumor and where the objective is to predict whether the tumor is benign or malignant. The dataset contains 100 samples, 8 independent features ( radius , texture , perimeter , area , smoothness , compactness , symmetry and fractal_dimension ) and one dependent feature ( diagnosis_result , the target feature).

The target feature contains 2 categories M = Malign and B = Benign . For practical purposes, we have coded the categories as follows: 0 = Malign and 1 = Benign.

The model

For this problem, we have implemented and optimized a model based on Random Forest obtaining an accuracy of 92% in the test set. The classifier implementation is shown in the following code snippet.

In the code snippet above, we implement a Random Forest based classifier which has been optimized using the Optuna library. From this point on, we already have a model that makes predictions which we intend to interpret with SHAP, so let’s see how we do it!

Interpretation: Shapley values

Suppose we have 2 samples that we intend to interpret, a sample that belongs to class 1 ( Benign ) and another sample that belongs to class 0 ( Malign ), which are shown in Figures 5 and 6 respectively.

Figure 5. Positive sample | Class = Benign
Figure 5. Positive sample | Class = Benign
Figure 6. Negative sample | Class = Malign
Figure 6. Negative sample | Class = Malign

Since our ML model is decision tree based, to obtain Shapley values we will use TreeShap. Therefore, we first need to import the library and initialize the explainer by passing as parameter our classifier, as shown in the next code snippet:

Next, we need the samples for which we want to calculate the Shapley values. In this case, I extract from the training dataset the samples that correspond to those shown in figures 5 and 6 respectively (it is important to mention that, for the purpose of this explanation, I am using one sample at a time, however the full dataset can also be used).

Just to check, the predictions for each sample are:

To calculate the Shapley values, we use the shap_values method that we extend from explainer. The argument it receives is the sample that we intend to interpret:

And there we have it, the Shapley values! But wait a minute, how do we interpret these values? why are they printed from index [1] ? Well, let’s see this in detail.

First, the variables shap_values_positive and shap_values_negative contain the Shapley values corresponding to each feature for both the positive and negative class, therefore, shap_values_positive[0] contains the Shapley values of each feature with respect to class 0 (or Malign ) and shap_values_positive[1] contains Shapley values of each feature with respect to class 1 (or Benign ), the same applies for shap_values_negative. For practicality, we use the results with respect to the class 1 .

Then, for both the positive and the negative samples, we obtain a list of 8 values, which correspond to the Shapley values for each feature. In the following figure, we can see more clearly each sample with its respective Shapley value:

Figure 7. Shapley values for each feature for each positive and negative sample
Figure 7. Shapley values for each feature for each positive and negative sample

Do you remember the essence of the Shapley values explained in the first section? Well, exactly the same thing happens here, features with high Shapley values have a greater impact, features with low Shapley values have less impact with respect to the prediction. We can also interpret Shapley values as, features with high shap values push towards one class and low shap values push towards the other class.

From the positive sample, we see that the features with the highest Shapley values are perimeter , compactness and area . From the negative sample, the features with the lowest Shapley values are perimeter , area and compactness .

As we mentioned, the shap library also provides tools for generating plots. Let’s see how to plot a called force_plot for our positive and negatie samples:

The resulting plots for each sample are:

Figure 8. Force plot for positive sample | Class = Benign | Image by author
Figure 8. Force plot for positive sample | Class = Benign | Image by author
Figure 9. Force plot for negative sample | Class = Malign | Image by author
Figure 9. Force plot for negative sample | Class = Malign | Image by author

As we can see from both plots, the impact of each feature corresponds to the Shapely values described in Figure 7. In red we see high Shapley values, that is, they push towards class 1 (or Benign), in blue we see low Shapley values, that is, they push towards class 0 (or Malign ). The force_plot is a great visualization to understand the impact of each feature on a specific sample for a given prediction.

The force_plot() is a great visualization to understand the impact of each feature on a specific sample, likewise the shap library provides various types of visualizations to favor the interpretation of Shapley values. For example, the summary_plot() method provides information about the feature importance as well as the impact of the Shapley value by varying the value of each feature.

For this example, the summary_plot() is given by:

generating the following plot:

Figure 10. Summary plot
Figure 10. Summary plot

On the left side we observe each feature ordered according to its importance. The perimeter feature being the most important and texture feature being the least important. The color represents the values that each feature can take, red for high values and blue for low values. Therefore, for the feature perimeter , if the values are high (red) the Shapley values will be low and consequently it will be pushing towards class 0 (or Malign ), otherwise, when the values are low (blue) the Shapley values they will be high that consequently they will be pushing towards class 1 (or Benign ). This can be verified with the results shown in Figures 8 and 9 respectively.

Another important observation is that the perimeter , compactness and area features are the ones that generate the most impact on the predictions given their associated Shapely values, that is, their Shapley values are very high or very low. On the other hand, the rest of the features do not generate as much impact because their associated Shapley values are closer zero.

A variation of the plot summary_plot() , presents in the form of bars the impact of each feature, such a plot can be obtained with:

generating the following plot:

Figure 11. Summary plot, bars
Figure 11. Summary plot, bars

As in figure 10, in figure 11 we can see each feature ordered according to impact with respect to its associated Shapley values. In the same way, we observe that the features perimeter , compactness and area are the ones that have the greatest impact on the model.

The complete project, with the examples shown in this article, can be found at: https://github.com/FernandoLpz/SHAP-Classification-Regression

And after several cups of coffee, for now we have reached the end!

Conclusion

In this blog we saw what SHAP (Shapley Additive Explanations) is.

In the first section we talked about the origin and the interpretation from the Shapely values. In the second section we learned what SHAP is, how it works and its support based on LIME and Shapely values for the interpretability of ML models. Finally in the third section we saw how to use the shap library and we show an example to interpret the results returned by the shap library.

The shap library provides a large number of tools to visualize and interpret ML models. In this blog we only saw a few examples. It is recommended to take a look at the documentation to get the maximum benefit from this great library.

"Read, learn, write, share. Repeat"

  • Fernando López Velasco.

References

[1] A Value for n-Person Games. L.S. Shapley. 1953

[2] A Unified Approach to Interpreting Model Predictions

[3] "Why Should I Trust You?" Explaining the Predictions of Any Classifier

[4] Breast Cancer Wisconsin (Diagnostic)


Related Articles