Difference between linear and logistic regression.

问题:

According to what I have understood linear regression predicts the outcome which can have continuous values whereas logistic regression predicts outcome which is discrete. So, according to me Logistic Regression is similar to a classification problem so why is called Regression? May be I am understanding something wrong but can't figure out the exact answer to this question.

I saw some previous links also related to their difference but it does not clear my question: What is the difference between linear regression and logistic regression?

回答1:

There is a strict link between linear regression and logistic regression.

With linear regression you're looking for the k_i parameters:

h = k₀ + Σ k_i ˙ X_i = K^t ˙ X

With logistic regression you've the same aim but the equation is:

h = g(K^t ˙ X)

Where g is the sigmoid function:

g(w) = 1 / (1 + e^-w)

So:

h = 1 / (1 + e^{-Kt ˙ X})

and you need to fit K to your data.

Assuming a binary classification problem, the output h is the estimated probability that the example x is a positive match in the classification task:

P(Y = 1) = 1 / (1 + e^{-Kt ˙ X})

When the probability is greater than 0.5 then we can predict "a match".

The probability is greater than 0.5 when:

g(w) > 0.5

and this is true when:

w = K^t ˙ X ≥ 0

The hyperplane:

K^t ˙ X = 0

is the decision boundary.

In summary:

• logistic regression is a generalized linear model using the same basic formula of linear regression but it is regressing for the probability of a categorical outcome.

This is a very abridged version. You can find a simple explanation in these videos (third week of Machine Learning by Andrew Ng).

You can also take a look at http://www.holehouse.org/mlclass/06_Logistic_Regression.html for some notes on the lessons.

回答2:

Logistic regression falls under the category of supervised learning.It measures the relationship between categorical dependent variable and one or more independent variables by estimating probabilities using logistic/sigmoid function. Logistic regression is a bit similar to linear regression or we can see it as a generalized linear model. In linear regression we predict output y based on a weighted sum of input variables.

y=c+ x1*w1 + x2*w2 + x3*w3 + .....+ xn*wn

The main purpose of linear regression is to estimate values of c,w1,w2,...,wn and minimize the cost function and predict y.

Logistic regression also does the same thing but with one addition. It pass the result through a special function called logistic/sigmoid function to produce the output y.

y=logistic(c + x1*w1 + x2*w2 + x3*w3 + ....+ xn*wn)

y=1/1+e[-(c + x1*w1 + x2*w2 + x3*w3 + ....+ xn*wn)]

回答3:

As explained earlier,logistic regression is a generalized linear model using the same basic formula of linear regression but it is regressing for the probability of a categorical outcome.

As you can see, we get similar type of equation for both linear and logistic regression. Difference lies in fact that linear regression give continous values of y for given x where logistic regression also gives continous values of p(y=1) for given x which is coverted later to y=0 or y=1 based on threshold value(0.5).