How to identify the best suited algorithm for classification?(Discriminant Analysis vs Logistic Regression)

classification
machine_learning

#1

Hello everyone,

Could anyone please explain the criterion for selecting LDA(Linear Discriminant Analysis) over Logistic Regression in certain kind of problems? While logistic regression maximises likelihood, discriminant analysis uses Bayes theorem to find the probabilities, and ultimately the discriminant function. But what I cannot understand here is how to identify which model would be more effective in which case?

Thanks in advance!


#2

@shashwat.2014
For the cases where we are using wide data means lesser observation with respect to the independent variables then we can use LDA instead of logistic regression.Because maximum likelihood estimation of the logistic model is well-known to suffer from small-sample bias.It may not capture the certain trends between dependent and independent variables for small sample size.In these cases, I think bayesian classifier is better.
Thanks,
Tapojyoti


#3

Thanks!


#4

There are other ways to understand where to use LDA over Logistic Regression.

  1. Whenever there is a big difference between the classes, then we use LDA over Logistic regression.
  2. Whenever you have less number of observations and the observations are normally distributed then we use LDA over Logistic regresison.
  3. Anytime you have more than two classes, then there is a slighter chance of LDA working better than Logistic regression.

(Source:ISLR)

I hope this helps!