Naive Bayesian and zero-frequency issue
algorithm, bayesian, machine-learning, spam-prevention
Solution
Still add one. The reason: Naive Bayes models `P("free" | spam)` and `P("free" | ham)` as being completely independent, so you want to estimate the probability of each completely independently. The Laplace estimator you're using for `P("free" | spam)` is `(count("free" | spam) + 1) / count(spam)`; `P("ham" | spam)` is the same.
If you think about what it would mean to not add one, it wouldn't really make sense: seeing "free" one time in ham would make it less likely to see "free" in spam.
Problem
I think I've implemented most of it correctly. One part confused me: The zero-frequency problem: Add 1 to the count for every attribute value-class combination (Laplace estimator) when an attribute value doesn’t occur with every class value. Here's some of my client code: ``` //Clasify string text = "Claim your free Macbook now!"; double posteriorProbSpam = classifier.Classify(text, "spam"); Console.WriteLine("-------------------------"); double posteriorProbHam = classifier.Classify(text, "ham"); ``` Now say the word 'free' is present in the training data somewhere ``` //Training classifier.Train("ham", "Attention: Collect your Macbook from store."); *Lot more here* classifier.Train("spam", "Free macbook offer expiring."); ``` But the word is present in my training data for category 'spam' only not in 'ham'. So when I go to calculate posteriorProbHam what do i do when I come across the word 'free'.