Using linear regression for a classification problem

linear_regression
r

#1

I was working on a data set that has different attributes related to Banking

Here the dependent variable(RESP) is categorical, however most of the other factors are of numeric type. Hence I wanted to apply linear regression for it instead of logistic regression, since most of the independent variables are of numeric type. Can we use linear regression for a classification problem? If yes, how do we convert the answers into 0 or 1? And isn’t it theoretically absurd?


About using Linear Regression with different types
#2

@Corporate_Cowboy - Please look at this post

http://discuss.analyticsvidhya.com/t/why-is-logistic-regression-required-why-not-linear-regression/6620

Hope this helps!

Regards,
HInduja


#3

@Corporate_Cowboy,
Applying linear regression for classification is not an absurd idea but logistic regression or other classification methods are preferred over linear regression.
You can apply linear regression for classification by assigning a threshold, given below is an example from an online course by Andrew NG where he fitted a line to the data set and used .5 as threshold for classification.

Although you should try to look at other classification techniques.
Hope this helps !!
Regards,
Syed Danish


#4

Hy @syed.danish

This is a very special case and I am sure Andrew mentioned it. Check the residual of most case and you will out why linear regression is not adequate for classification.

Alain


#5

Thank you all for the help.

The discussion from that link was really helpful @hinduja1234. There are some articles given over there which proved to be really useful in understanding the concept. @syed.danish I saw that very video and was able to understand the concept.

I think that it is safe to say that when all the assumptions of linear regression are met, we tend to find linear regression suitable for a classification problem. Correct me if I am wrong.


#6

When classification is binary as in this case it is always good to prefer logistic reg. Linear reg will do its job but the accuracy will be less compared to log reg.


#7

Thanks @Brijesh.


#8

The decision to select Linear OR Logistic regression is not based on the type of predictor variables., Since your dependent variable is categorical with two classes, you will go with logistic regression.