A decision tree asks a sequence of yes/no questions about the features, each one splitting the data to make the two resulting groups as "pure" as possible, mostly one label or the other. For malware detection, a tree might learn, entirely on its own from data, that "does this file request the CreateRemoteThread API" is a far better first question than "is the file larger than 1 MB."
- At the current node, consider every feature and every possible threshold.
- For each candidate split, measure how much it reduces impurity (Gini impurity or entropy) in the two resulting groups.
- Choose the split that reduces impurity the most.
- Repeat inside each child node, until a stopping rule is reached (maximum depth, minimum samples per leaf, or no further useful split exists).
- A leaf's prediction is simply the majority label of the training examples that landed there.
decision_tree_phishing.py
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
# X: engineered URL features (length, digit ratio, has-IP-address,
# subdomain count, HTTPS flag, ...); y: 1 = phishing, 0 = legitimate
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=7, stratify=y
)
tree = DecisionTreeClassifier(max_depth=5, min_samples_leaf=10, random_state=7)
tree.fit(X_train, y_train)
print(f"Test accuracy: {tree.score(X_test, y_test):.3f}")
Caution: A Tree That Memorises Is Not a Tree That Generalises
An unconstrained tree, grown without max_depth or min_samples_leaf, will keep splitting until every leaf is perfectly pure, including leaves with a single training example. That tree has not learned a pattern; it has memorised the training set, and its performance on new data is usually far worse than its training accuracy suggests. This is overfitting, and it is the single most common mistake in applied ML.
Try it yourself: Spot the Overfit
Two trees are trained on the same phishing-URL data. Tree A uses max_depth=5, min_samples_leaf=10 and scores 0.89 on training data and 0.86 on test data. Tree B uses no depth limit at all and scores 0.999 on training data and 0.71 on test data. Which tree would you actually deploy, and what single number in this comparison tells you why?