ml-for-cyber-securitymlcsaccuracyclass-imbalance

Why Accuracy Lies on Security Data

A classifier that has learned absolutely nothing scores 99.8 percent accuracy on a realistic fraud dataset. That number sounds excellent and is worthless, because it catches zero fraud, the entire point of building the model in the first place.

Ashish Revar25 October 20265 min read

Return to the class-imbalance chart first raised in Unit 1 and sharpened here: a realistic credit-card fraud dataset is roughly 99.8 percent legitimate transactions. A classifier that predicts "legitimate" for every single transaction, one that has learned nothing at all, scores 99.8 percent accuracy. That number sounds excellent and is worthless, because it catches zero fraud, the entire point of building the model in the first place.

A realistic fraud dataset on a logarithmic scale. A model that always predicts "legitimate" would score above 99.8% accuracy while catching no fraud at all, which is exactly why accuracy alone cannot be trusted on data shaped like this

Key takeaway

This closing unit explains why that number can be close to meaningless on real security data, builds the metrics that actually matter, confronts the uncomfortable fact that a trained model can be deliberately fooled, and names the public benchmark datasets this field builds on.

Discussion

Loading comments…