Recursive Feature Elimination (RFE) in Regression and Classification Models

Using the RFECV() function in Scikit-learn and Yellowbrick

Photo by Visax on Unsplash

Now all features in the dataset contribute the same to machine learning models.

We can remove the unwanted features from the model by using special feature selection techniques. Recursive feature elimination (RFE) is one of them. Doing so will give you the following benefits.

• Reduce the complexity of the model: This will enhance the training speed and interpretability of the model.
• Remove unnecessary noise generated from less important features: This will regularize the model and prevents overfitting.
• Remove dependencies and collinearity between the input features: This will also regularize the model and prevents overfitting.

In short, recursive feature elimination (RFE) recursively eliminates one feature or a small set of features at a time using cross-validation (CV).

RFE is also a dimensionality reduction method as it reduces the number of features in the model by removing unwanted features.