Logistic Regression for Multiclass Classification — 3 Strategies You Need to Know

One-vs-Rest, One-vs-One and Multinomial Methods

Photo by Henry Lorenzatto on Unsplash

By design, logistic regression models automatically handle binary classification problems in which the target vector (label column) has only two classes.

However, three extensions to logistic regression are available to use logistic regression for multiclass classification in which the target vector has more than two classes.

These extensions include:

• One-vs-Rest (OvR) multiclass strategy
• One-vs-One (OvO) multiclass strategy
• Multinomial method

We’ll begin with the One-vs-Rest (OvR) multiclass strategy.

One-vs-Rest (OvR) multiclass strategy

When there are more than two classes in the target vector, the one-vs-rest strategy allows logistic regression to train a separate model for each class comparing with all the remaining classes. Therefore, this is also known as the One-vs-All (OvA) strategy.

This method creates one logistic regression model for each class against all other classes. If the target vector has four classes (e.g. Cat, Dog, Monkey, Bear), this strategy will create four separate models in the following way.

• Model 1: Cat vs [Dog, Monkey, Bear]
• Model 2: Dog vs [Cat, Monkey, Bear]
• Model 3: Monkey vs [Cat, Dog, Bear]
• Model 4: Bear vs [Cat, Dog, Monkey]

One way to implement the OvR strategy with logistic regression is to specify the multi_class="ovr" argument.

# Create one-vs-rest logistic regression instance
from sklearn.linear_model import LogisticRegression

model = LogisticRegression(multi_class="ovr")

# Train the model
model.fit(X_train, y_train)

# Make predictions
y_pred = model.predict(X_test)

Another way to implement the OvR strategy with logistic regression is to use the OneVsRestClassifier API provided by the Scikit-learn library.

from sklearn.linear_model import LogisticRegression
from sklearn.multiclass import OneVsRestClassifier

#…