Model Explainability, Revisited: SHAP and Beyond
The rapid rise of large language models has dominated much of the conversation around AI in recent months—which is understandable, given LLMs’ novelty and the speed of their integration into the daily workflows of data science and ML professionals.
Longstanding concerns around the performance of models and the risks they pose remain crucial, however, and explainability is at the core of these questions: how and why do models produce the predictions they offer us? What’s inside the black box?
This week, we’re returning to the topic of model explainability with several recent articles that tackle its intricacies with nuance and offer hands-on approaches for practitioners to experiment with. Happy learning!
- At the core of any explainability challenge is the question of which features in your data contribute the most to a model’s prediction. Khouloud El Alami’s introduction to feature-importance analysis with SHAP is a beginner-friendly resource, based on the author’s research project at Spotify.
- If you’ve already worked with SHAP in the past and are looking to expand your toolkit, Conor O'Sullivan offers a hands-on guide to handling more specialized use cases, namely how to display SHAP plots for classification problems and aggregate SHAP values for multi-class targets.
- For a fresh perspective on the possibilities that model explainability opens up, don’t miss Diksha Sen Chaudhury’s recent writeup on a project that brings together healthcare data and machine learning. Diksha’s goal was to show how using SHAP can make a model not just interpretable, but also useful for researchers who’d like to benchmark results against the findings in medical literature.
- As Vegard Flovik aptly puts it, “for applications within safety-critical heavy-asset industries, where errors can lead to disastrous outcomes, lack of transparency can be a major roadblock for adoption.” To address this gap, Vegard provides a thorough guide to the open-source Iguanas framework, and shows how you can leverage its automated rule-generation powers for increased explainability.
- While SHAP values have proven beneficial in many real-world scenarios, they, too, come with limitations. Samuele Mazzanti cautions against placing too much weight (pun intended!) on feature importance, and recommends paying equal attention to error contribution, since “the fact that a feature is important doesn’t imply that it is beneficial for the model.”
We know that the beginning of September is a hectic time of year for many of you, but if you have a bit more time to spare, you can’t go wrong with any of our other recommended reads this week:
- If you’re in a data science bootcamp right now—or are thinking of attending one in the future—Alexandra Oberemok’s comprehensive guide to making the most of the experience is a must-read.
- Runners, this one’s for you: barrysmyth’s new deep dive explores marathon data to assess different strategies for optimizing your performance.
- For his debut TDS article, Christian Burke takes us behind the scenes of an innovative MOMA generative-AI art project in which he played a key part.
- Olga Chernytska shared a new installment in her excellent “Building Better ML Systems” series, this time focusing on all things baselines, metrics, and test sets.
- Not sure how to deal with missing data? Miriam Santos provides a one-stop resource on this perennial issue, and explains how to identify and mark missing values in real-world datasets.
- If you’re looking to sink your teeth into a detailed technical explainer, Antonieta Mastrogiuseppe’s overview of the gradient descent algorithm is clear and well-executed.
Thank you for supporting the work of our authors! If you enjoy the articles you read on TDS, consider becoming a Medium member — it unlocks our entire archive (and every other post on Medium, too).
