Zero-frequency problem

naive_bayes

#1

I came across the Zero-frequency problem during the study of naive Bayes. But i am unable to understand what it is.


#2

Hi,

One of the disadvantage with Naive-Bayes is that if you have no occurrences of a class label and a certain attribute value together then the frequency-based probability estimate will be zero.

And this will get a zero when all the probabilities are multiplied.

Regards,
Imran


#3

The solution to the zero-frequency problem would be to shift your zero frequency to slightly higher. So, if you have counts of some occurrence in your data use Laplacian smoothing. Some of the examples can be found here:
http://www.cs.upc.edu/~belanche/Docencia/mineria/English-september-2008/Practical-work/Labo-NBayes-kNN.R

Basically, use the laplace option in naiveBayes:

model <- naiveBayes(Class ~ ., data = HouseVotes84, laplace = 3)