Evasion attacks the model at prediction time; a poisoning attack corrupts it earlier, during training. This topic reimplements the chaff-injection technique from Chio and Freeman's companion notebook on a fresh dataset, training a real neural network classifier and then poisoning it with partial_fit, scikit-learn's incremental-training method.
poison_mlp_classifier.py
from sklearn.neural_network import MLPClassifier
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=200, n_features=2, n_informative=2,
n_redundant=0, weights=[0.5, 0.5], random_state=17)
clf = MLPClassifier(hidden_layer_sizes=(8,), max_iter=800, random_state=123)
clf.fit(X[:100], y[:100]) # initial, clean training
# the attacker injects a small cluster of points, all labelled with
# their preferred (incorrect) class, near the existing decision boundary
chaff_X = np.column_stack([np.linspace(-2, -1, 12), np.full(12, 0.1)])
chaff_y = np.ones(12)
for _round in range(6):
clf.partial_fit(chaff_X, chaff_y) # the model keeps learning online
A real MLP classifier's decision surface, before and after six rounds of chaff injection near the injected points, bulges measurably around them. A more patient or better-targeted attacker, injecting more chaff over more rounds, bends it further.
Caution: Why Online, Continuously Retrained Models Are More Exposed
This attack only works because the model keeps learning after deployment, through partial_fit, exactly the property that makes a model useful for adapting to new, legitimate traffic patterns without a full retrain. Any system that retrains automatically on live, unvetted data, a spam filter learning from user "not spam"/"spam" clicks, a fraud model retrained nightly on the previous day's transactions, carries this exact exposure. The 2016 Microsoft Tay incident (Unit 1's timeline) is this same mechanism, running on language instead of two-dimensional points.
Try it yourself: Make the Poisoning Attack Stronger
In the code above, increase the chaff cluster from 12 points to 40, and the number of partial_fit rounds from 6 to 20. Regenerate the before/after decision boundary and compare how much further it shifts. This directly demonstrates that poisoning's effectiveness scales with how much of the (re)training data the attacker controls, the variable every defence against poisoning ultimately tries to limit.