SHAP for Feature Selection and HyperParameter Tuning

Use SHAP for optimal Feature Selection while Tuning Parameters

Marco Cerliani
Towards Data Science

Photo by Lanju Fotografie on Unsplash

Feature selection and hyperparameter tuning are two important steps in every machine learning task. Most of the time they help to improve the performances but with a drawback to be time expensive. The more the parameter combinations, or the more accurate the selection process, the higher the duration. This is a physical limit that actually we can’t defeat. What we can do is leverage the best from our pipeline. We face different possibilities, two of the most convenient are:

  • Combine the tuning and the selection of features;
  • adopt SHAP (SHapley Additive exPlanations) to make the whole procedure more generalizable and accurate.

Combining the tuning process with the optimal choice of features may be a need of every ranking-based selection algorithm. A ranking selection consists of iteratively dropping the less important features while retraining the model until convergence is reached. The model used for feature selection may differ (in parameter configuration or in the type) from the one used for final fitting and prediction. This may result in suboptimal performances. This is the case for example of RFE (Recursive Feature Elimination) or Boruta, where the features, selected…

Create an account to read the full story.

The author made this story available to Medium members only.
If you’re new to Medium, create a new account to read this story on us.

Or, continue in mobile web

Already have an account? Sign in