Reinforcement Learning

Reinforcement learning trains an agent by letting it act and rewarding the results. There are no correct answers to copy, only outcomes that turn out better or worse.

The hard part is that rewards arrive late. Win a chess game and every move contributed something, but which ones actually mattered? Working that out is most of the difficulty.

Think of it like this. Think of learning to ride a bike. Nobody hands you the correct handlebar angle for each moment. You try things, you fall, and eventually you stop falling without ever being told the rule.

How it works

• The agent observes a state. The current situation.
• It takes an action from the options available.
• The environment answers with a reward and a new state.
• The loop repeats, with the agent trying to maximise reward over the long run rather than at each step.
• Exploration versus exploitation. Repeat what worked, or try something new that might work better. Getting this balance wrong is the classic failure.

Common mistakes

• "Just define the reward and it learns." Reward design is the hardest part. Agents reliably find behaviour that scores well and defeats your intent entirely.
• "It is supervised learning with rewards." Supervised learning is corrected on every example. Here the feedback is delayed, sparse, and only tells you the score, not the right move.
• "Exploration is optional." An agent that only repeats what worked settles for the first decent strategy it stumbles into.
• "It works in simulation, so it works." Policies trained in simulation routinely fail in the real world, because the simulation was subtly wrong.