Reading about evasion is one thing; running one against a classifier you trained yourself, in this course's own code, is another. This topic builds a small web application firewall (WAF), a classifier that scores an HTTP query string as malicious or benign, then evades it using exactly the technique Chio and Freeman demonstrate in their companion code: find the specific token the model has learned to associate most strongly with "benign," then exploit that association directly.
waf_classifier.py
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
# malicious: SQLi and XSS payloads; benign: ordinary query strings
vectorizer = TfidfVectorizer(analyzer="char", ngram_range=(1, 3), sublinear_tf=False)
X = vectorizer.fit_transform(queries)
clf = LogisticRegression(class_weight="balanced", max_iter=2000, C=5.0)
clf.fit(X, labels)
waf = Pipeline([("vectorizer", vectorizer), ("classifier", clf)])
payload = "<script>alert(document.cookie)</script>"
print(f"P(malicious) = {waf.predict_proba([payload])[0, 1]:.4f}")
exploit_learned_bias.py
import numpy as np
# term_influence combines each character n-gram's rarity (idf) with how
# strongly the model's coefficient pushes toward benign (negative) or
# malicious (positive)
term_influence = vectorizer.idf_ * clf.coef_[0]
vocab = {v: k for k, v in vectorizer.vocabulary_.items()}
most_benign_idx = np.argmin(term_influence)
most_benign_term = vocab[most_benign_idx]
evasive_payload = payload + "/" + most_benign_term * 150
print(f"Most benign-pulling n-gram: {most_benign_term!r}")
print(f"P(malicious) after padding = {waf.predict_proba([evasive_payload])[0, 1]:.4f}")
Sample output
P(malicious) = 0.9594
Most benign-pulling n-gram: 'y'
P(malicious) after padding = 0.2836
Key takeaway
The payload's score dropped from 0.9594 to 0.2836, crossing the 0.5 decision threshold, so the classifier's verdict genuinely flipped from malicious to benign, purely by appending a single padding character the model had learned to associate with legitimate queries. The XSS payload itself is untouched and would still execute exactly as written if a vulnerable page reflected it. This is the same mechanism, in miniature and fully reproducible on a laptop, that Unit 2's Skylight Cyber case study describes at production scale against a commercial AI antivirus engine.
Try it yourself: Find a Different Bias
The exploit above used np.argmin(term_influence) to find the single most benign-pulling character n-gram. Modify the code to instead find and print the five most benign-pulling n-grams. Do any of them combine into a recognisable English word or common URL fragment, and does that suggest anything about what the training data's benign examples had in common?