Naive Bayes
A fast probabilistic classifier that assumes features are conditionally independent given the class.
Definition
Naive Bayes applies Bayes' theorem with the simplifying assumption that features are independent given the class label. This assumption is usually false, yet the classifier is often accurate and extremely fast to train.
Despite its crude independence assumption, it often ranks classes correctly even when its probability estimates are off, because errors in the assumed likelihoods tend to affect all classes similarly. This is why it remains a strong, fast baseline for high-dimensional text problems.
Its speed and low data requirements make it a natural first model and a useful component in larger systems, such as an initial filter before a heavier classifier. Smoothing, adding a small count to every category, prevents a single unseen feature value from zeroing out a class probability entirely, a simple fix that is essential for the method to work on real text.
It estimates the probability of each class from the product of per-feature likelihoods and picks the most probable class.
Variants
- Gaussian: continuous features modeled as normal.
- Multinomial: count features, common in text.
- Bernoulli: binary presence/absence features.
Why it matters
Naive Bayes needs little data, trains in one pass, and is a strong baseline for text classification and spam filtering. Its probability estimates can be poorly calibrated because the independence assumption is violated, but its class rankings are frequently good.
Fusion connection
As a lightweight baseline, Naive Bayes offers a quick sanity check for categorizing diagnostic signals before Kronos commits to heavier models.