ML for Cyber Security
ML for Cyber Security
Sign in to save your progress, earn points, and unlock module exams.
Unit 4 of MLCS
The Markov decision process, Q-learning, adaptive honeypots that decide rather than just detect, and rate-limiting network traffic as a sequential decision problem.
The dog-training analogy for reinforcement learning, and the four pieces every RL problem is built from: state, action, reward and policy, illustrated with an adaptive honeypot example.
The five-element MDP tuple (S, A, P, R, gamma) that frames nearly every reinforcement learning problem, and the Markov property that makes the problem tractable.
The five-step Q-learning update loop, the update rule itself, and the exploration-versus-exploitation trade-off every practical RL agent has to balance with an epsilon-greedy policy.
A complete, runnable Q-learning agent deciding how deeply to engage an attacker session, a real reward-curve chart showing the agent learning over 300 episodes, and why static honeypots like Cowrie eventually get fingerprinted.
A second worked Q-learning agent for adaptive rate-limiting with a continuous, discretised state space, why large state spaces need Deep Q-Networks instead of a table, moving target defence as an active research area, and the risk of reward-hacking.
A one-page summary table of every topic in this unit, seven self-study questions to test understanding, and a lab exercise changing the honeypot's reward function and watching the learned policy change.
Assessments
ML for Cyber Security: Complete Assessment
Start final exam →