SHAP for Feature Selection and HyperParameter Tuning
Use SHAP for optimal Feature Selection while Tuning Parameters
Feature selection and hyperparameter tuning are two important steps in every machine learning task. Most of the time they help to improve the performances but with a drawback to be time expensive. The more the parameter combinations, or the more accurate the selection process, the higher the duration. This is a physical limit that actually we can’t defeat. What we can do is leverage the best from our pipeline. We face different possibilities, two of the most convenient are:
- Combine the tuning and the selection of features;
- adopt SHAP (SHapley Additive exPlanations) to make the whole procedure more generalizable and accurate.
Combining the tuning process with the optimal choice of features may be a need of every ranking-based selection algorithm. A ranking selection consists of iteratively dropping the less important features while retraining the model until convergence is reached. The model used for feature selection may differ (in parameter configuration or in the type) from the one used for final fitting and prediction. This may result in suboptimal performances. This is the case for example of RFE (Recursive Feature Elimination) or Boruta, where the features, selected…