To make the idea concrete before any theory, compare two ways of deciding whether an email is spam, using a labelled spam corpus whose class balance is plotted below. The first approach is a blacklist: a fixed set of sender addresses and keywords, checked with simple string matching.
spam_blacklist.py
BLOCKED_SENDERS = {"promo@dealsnow.biz", "winner@luckydraw.ru"}
BLOCKED_WORDS = {"viagra", "lottery", "wire transfer"}
def is_spam_blacklist(sender: str, subject: str, body: str) -> bool:
if sender.lower() in BLOCKED_SENDERS:
return True
text = f"{subject} {body}".lower()
return any(word in text for word in BLOCKED_WORDS)
This works perfectly for the first email it was written for, and fails the moment a spammer changes one letter. The second approach trains a Naive Bayes classifier on the frequency with which each word appears in spam versus legitimate mail, so a message never seen before is scored by how closely its vocabulary resembles the spam it has learned from.
naive_bayes_spam.py
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
# messages: list[str], labels: list[int] (1 = spam, 0 = legitimate)
X_train, X_test, y_train, y_test = train_test_split(
messages, labels, test_size=0.25, random_state=42, stratify=labels
)
vectorizer = CountVectorizer(stop_words="english", min_df=2)
X_train_vec = vectorizer.fit_transform(X_train)
X_test_vec = vectorizer.transform(X_test)
clf = MultinomialNB()
clf.fit(X_train_vec, y_train)
print(classification_report(y_test, clf.predict(X_test_vec),
target_names=["legitimate", "spam"]))
Sample output
precision recall f1-score support
legitimate 0.98 0.99 0.98 1207
spam 0.96 0.94 0.95 453
accuracy 0.97 1660
macro avg 0.97 0.96 0.97 1660
weighted avg 0.97 0.97 0.97 1660
ML concept: Naive Bayes, in One Paragraph
Naive Bayes asks: given the words in this message, which label, spam or legitimate, makes those words most probable, according to what I saw in training? It is called "naive" because it assumes every word's presence is independent of every other word's presence, which is not literally true of real language, but works remarkably well in practice for exactly this reason: it needs very little data to start making good predictions, and it is fast enough to score a message in microseconds.
Try it yourself: Where Would the Blacklist Fail First?
A spammer rewrites "viagra" as "v1agra" and starts sending from a fresh address every day. Trace through is_spam_blacklist above and identify the exact line that now fails silently. Would the Naive Bayes classifier trained in the second script be fooled the same way? Why or why not?