Unit 5 of MLCS

Malware and Phishing Detection

Feature engineering from network traffic and malware binaries, lexical URL features, NLP features for phishing email detection, and malware detection models in practice.

Topics

5.1

Feature Engineering Is the Real Skill

Why a strong feature set carries a simple classifier further than a weak feature set carries a sophisticated one, and the three tests, discriminative, available at decision time, robust to trivial evasion, every feature in this unit is checked against.

5 min

5.2

Engineering Features from Network Traffic

Flow-level features, duration, packets per second, packet size statistics, byte ratios, and TCP flag counts, that summarise raw packets into the fixed numeric vector a model can actually learn from, with a runnable summarisation function.

5 min

5.3

Engineering Features for Malware Classification

Static analysis features, imported API calls, section entropy, PE header anomalies and string artefacts, and a runnable Shannon entropy function, the single most informative static feature a malware classifier can have.

5 min

5.4

Malware Detection Models in Practice

The five-step production malware-detection pipeline, from static and dynamic feature extraction through scoring, human review of low-confidence predictions, and regular retraining, plus the WannaCry and Mirai case studies that changed how defenders think about features.

5 min

5.5

Lexical URL Features: TF-IDF as an Alternative to Hand-Engineering

Treating a URL as text and letting TF-IDF discover discriminative substrings automatically, a runnable URL tokeniser and TF-IDF classifier, and the trade-off between lexical and hand-engineered structured URL features.

5 min

5.6

NLP Features for Phishing Email Detection

A full TF-IDF plus logistic regression phishing-email pipeline with bigrams, why ngram_range=(1,2) lets the model separately learn phrases like urgent action, a real chart of the actual top ten words a trained model learned to associate with spam, and the SMS Spam Collection benchmark.

5 min

5.7

Combining Text and Structural Features for Phishing

Why pure text features miss structural signals, a real feature-importance chart showing sender-link mismatch and urgency-word count outranking raw word counts, the Business Email Compromise case study where there is no malware payload at all, and the five-step combined feature pipeline, plus feature leakage.

5 min

5.8

Unit 5 Quick Reference, Self-Study and Lab

A one-page summary table of every topic in this unit, seven self-study questions to test understanding, and a lab exercise building your own structural phishing feature and checking its feature-importance ranking.

6 min

Assessments

Unit Assessment

Malware and Phishing Detection

20 questions30 minPass: 60%

Start exam →

Comprehensive Assessment