Logistic Regression in Machine Learning
Introduction
One of the most widely used Machine Learning algorithms, under the category of Supervised Learning, is logistic regression. Using a specific set of independent variables, is used to predict the categorical dependent variable. It is used to solve classification problems, whereas linear regression is used to solve regression problems. The output of a categorical dependent variable is predicted by this regression. The result must therefore be a discrete or categorical value. It can be either True or False, Yes or No, 0 or 1, etc., but rather than providing the exact values of 0 and 1, it provides the probabilistic values that fall between 0 and 1.
Logistic Function (Sigmoid Function)
We require a function to transform this straight line so that values fall between 0 and 1.
Let,
Sigmoid function: σ(z) = 1/(1+e−z)
Linear model : ŷ = a0+a1x
Logistic regression model: ŷ = 1/(1+e-(a0+a1x))
We will obtain a line that remains between 0 and 1 after transformation. So, in contrast to linear regression, logistic regression results in an “S”-shaped curve.
Cost Function for Logistic Regression
The most significant classification metric based on probabilities is log loss. A lower log loss value indicates better predictions for any given problem.
where,
Here, yi stands for the actual class, and log(p(yi) is the class’s probability.
The probability of 1 is given by p(yi).
The probability of 0 is given by 1-p(yi).
Types of Logistic Regression
There are mainly three different types.
1. Binary logistic regression: When the dependent variable (Y) is binary in nature, the relationship between the dependent variable (Y) and the independent variable (X) is predicted using a statistical technique. The result, for instance, could be True/False, Success/Failure, 0/1.
2. Multinomial logistic regression: There may be three or more distinct unordered types of the dependent variable. For example “tiger”,”lion” and “elephant”.
3. Ordinal logistic regression: There can be three or more different ordered types of dependent variables, and the dependent variable has a meaningful order. For example “Low”, “medium” and “High”.
Also read: Implementation using sklearn
read: Prediction using Logistic regression
