What is the attribute independence problem in Naive Bayes classifier?




The Naive Bayes classifier infers that the probability
of a class label given data using a simplifying assumption that the attributes are independent
given the label.
What problem does it create and how do we solve it?



I’ll share my idea and experience in text mining using Naive Bayes Classifer. To understand the independence assumption consider this, the classifier does not take into consideration the position of the word that you are analysing relative to the positions of the other words in the collection, ideally “I think Naive Bayes Classifier is truly Naive but can be quite useful” will be similar to “Naive truly I Classifier…” because Naive Bayes just considers the tokens to calculate the presence or absence arriving at the probability classifier.

What I do is:

(a)Check the level of accuracy using Naive Bayes and consider how well it classifies
(b) I also tend to use a Parser (http://nlp.stanford.edu:8080/parser/) a variety of which is available in packages like R and Python. This method takes into account the relative positions and thereby assigns a POS tag, which can have a strong impact the way sentence is expressed.

Example -
I was like “what do you mean?” Here ‘like’ would be classified as Preposition
I like the movie alot. Here ‘like’ would be classified as Verb