Multinomial Logistic Regression from Scratch



Hi All,

there was an interesting article on building Logistic Regression classifier from scratch

However i need to build multinomial LR … how should this code be modified in order to achieve it from scratch



Hi @swati0205 ,

Multinomial logistic regression works in a little bit different way.

I am listing down the steps below for you to understand how it works. for you to understand lets take a scenario.

Suppose you have to predict the appettite(High/ Medium/ Low) of a person based on eating frequency, size of per meal and meal preference(veg/ non-veg/eggetarian). We will go ahead something like below.

  1. Clean the data(of-course), change the categorical independent variable into dummy variable and do EDA
  2. second step is to relevel the predictor variable appetite. this is we do to make one of choice(high/medium/low) as baseline
  3. You use multinom function(not glm in this case) everything else is same.
  4. after creating the first model with multinom function you get the coefficients and standard error of each independent variable.
  5. In multinom function, p-value is not automatically calculated so you have to calculate that from coefficients and standard errors (will provide you the link below for your understanding)
  6. After that your normal variable reduction goes with P-value and VIF.
  7. As a result of multinom function, you get coefficient of all choice of dependent but the one you use the baseline for.
  8. From coefficients, you have to calculate the probability function.
  9. this probability function will tell you the probability of each choice based on the test data you have.
  10. model evaluation is almost same a logistic regression.

Above the very high level explanation of it. If you want to understand more. Please follow below link for it. it has been explained beautifully.