Glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message when I run Logistic regression

r
logistic_regression

#1

(glm.fit: fitted probabilities numerically 0 or 1 occurred) warning message occured for Logistic regression.
Help me resolving this issue.


#2

@Ronit
Reproduce the entire problem ronit with code what you are doing. Asking a broad question won’t help you much.

Regards


#3

Hi @Ronit,

Please elaborate on the dataset that you are working with so that we can give a conclusive suggestion. This error means that the model is predicting absolute probabilities like 0 and 1.

If you feel that in the problem that you are dealing with can have such possibilities, it is advisable to leave this as it is and ignore the warning. However, if you feel that this should not be happening, you could try the following:-

  1. Use bayesglm in the arm package

fit <- bayesglm(y ~ x1 + x2+... , data=d, family="binomial")

  1. Penalize the coefficients of the logistic regression using glmnet.
    One of the less user-friendly aspects of glmnet is that you can only feed it matrices, not formulas as we’re used to. However, you can look at model.matrix and the like to construct this matrix from a data.frame and a formula.

fit2=glmnet(x,g2,family="binomial")

Just have a look at the documentations of these packages to see the complete syntax and execution.

Regards,
Shashwat


#4

Hi Shashwat, sorry for bringing out this post from dead. But I’m facing this problem and whatever solution I’ve found till date went over my head. Just to let you know, I’m a newbie to data science and taking part in my first analytics competition.

So here is the issue,

I have about 9 predictors and i’m trying to predict a student placement through these variables. Though I have already built my model using Forward selection and have achieved about 96% accuracy, i’m still wondering if I can enhance it by using other predictors that are creating the problem of “glm.fit: fitted probabilities numerically 0 or 1 occurred”. Based on your questions above, here are the answers:

  1. No the problem cannot have any such probabilities to have exact 0 or 1.

  2. As a result of this warning, all my model values are inflated. In fact all my p-values for predictors are coming out to be 1, strange isn’t it?

  3. I just tried the baysian glm and got following warning : “fitted probabilities numerically 0 or 1 occurred”

  4. I couldn’t understand that glmnet solution that you gave, actually I found the same solution somewhere else on net as well but couldn’t decipher it. Can you please explain it?

Please feel free to ask for any other details and I will provide it.

Awaiting your reply.

Thanks
gaurav


#5

Hi Gaurav

I am facing similar problem as yours. Were you able to find a solution for this warning ? Really appreciate your help.

Thanks
Pooja


#6

Hi Pooja,

Yes I was able to :slight_smile:

Based on my research, I found that this error occurs mostly due to following 2 reasons:

  1. When one of your IV’s are able to segregate the DV perfectly well. Means, for a particular value of IV’s you can always say that one of the two outcomes (of the DV) exists.

  2. When number of fisher scoring iteration (by defalut 25, I guess) are not enough for convergence.

What to do?

  1. Instead of complete logistic regression, try step wise logit and see which variable is creating the perfect separation.

  2. To increase the number of fisher scoring, try adding this to your existing code : “control=glm.control(maxit=50)”. Value of maxit can be changed ofcourse.

For a detailed information on this topic refer Dealing with perfect separation in logit.

Hope this helps.

Peace.