Analogy: Training a Dog
Think of training a dog. You do not hand the dog a labelled dataset of "good sit" and "bad sit." You let it try something, and when it sits on command, you give it a treat. Over many repetitions, the dog learns a policy, a mapping from situation (you say "sit") to action (sit down), that maximises treats over time. It never saw a single labelled example; it learned entirely from the consequences of its own actions.
That is reinforcement learning. An agent observes a state, takes an action, and receives a reward from the environment, then observes the next state and repeats. Over many episodes, the agent learns a policy, a rule for choosing actions, that maximises its total reward.
ML concept: State, Action, Reward, Policy
- State: everything the agent currently knows about the situation (for a honeypot: how long an attacker has been connected, what commands they have run so far).
- Action: a choice available to the agent (keep the session open, serve a fake error, terminate the connection).
- Reward: a number scoring how good that action turned out to be (information gathered, minus the cost of the attacker's time and the risk of detection).
- Policy: the learned rule mapping states to actions, chosen to maximise cumulative reward, not just the next reward.
Key takeaway
Every model in Units 2 and 3 answers a single question about a single piece of data: is this malicious, does this fit, how unusual is this? Neither one ever acts. Reinforcement learning is different: an agent takes an action, the environment responds, and the agent learns from the consequence over time. This unit builds the basics and applies them to a defender that adapts, not just classifies: the adaptive honeypot.