A single tree overfits easily and is sensitive to small changes in the training data. A random forest trains many trees, each on a random bootstrap sample of the training data and a random subset of features at each split, and lets them vote. No individual tree needs to be excellent; the forest's accuracy comes from the trees' errors being only weakly correlated with each other, so averaging cancels most of the noise.
Tool in focus: sklearn.ensemble.RandomForestClassifier
RandomForestClassifier is scikit-learn's implementation. The two parameters worth tuning first are n_estimators (how many trees; more is almost always better, at a compute cost) and max_depth (how deep each tree may grow; shallower trees generalise better but can underfit).
random_forest_phishing.py
from sklearn.ensemble import RandomForestClassifier
forest = RandomForestClassifier(
n_estimators=200, max_depth=4, random_state=7, n_jobs=-1
)
forest.fit(X_train, y_train)
print(f"Test accuracy: {forest.score(X_test, y_test):.3f}")
Case study: Endpoint Protection Goes Statistical
Through the 2010s, several commercial endpoint protection vendors, most publicly CrowdStrike and Cylance, replaced or supplemented signature databases with trained classifiers, often ensembles of trees, scoring a file's static features (imported APIs, section entropy, header structure) for maliciousness before the file ever runs. The approach catches variants of known malware families that share structural features even when every byte of their signature has changed. Unit 6 returns to this exact case study from the opposite direction: in 2019, researchers at Skylight Cyber showed that BlackBerry Cylance's AI engine could be fooled by appending unrelated strings from a benign video game executable to malicious files, moving one tested sample's maliciousness score from -852 to +999 on the product's own -1000 to +1000 scale. A statistical model is a genuine improvement over a static signature list, but it inherits a new failure mode a signature list never had.