Reinforcement Learning
Learning by trial and error through rewards — the way you'd train a pet, applied to software.
When not to use it
- When you have labelled examples. If you can show the right answer, supervised learning is dramatically cheaper and more stable. RL is for when you can only score outcomes, not demonstrate them.
- In the real world without a simulator. RL learns by failing, repeatedly. If each failure costs money, hardware, or trust, you can't afford the curriculum.
- When you can't specify the reward precisely. A misspecified reward doesn't fail loudly — it gets optimised, and you get exactly what you asked for.
Reach for something else instead
- Supervised learning whenever demonstrations exist.
- Bandits for the common case of choosing among options with feedback — simpler, well-understood, and enough for most recommendation and pricing problems.
- Classical optimisation and control for problems that have structure you already understand.
Sources & further reading
- Sutton & Barto, Reinforcement Learning: An Introduction — the book, free from the authors.
- Mnih et al. (2013), Playing Atari with Deep Reinforcement Learning — the result that made deep RL credible.
- Clark & Amodei (2016), Faulty Reward Functions in the Wild — reward hacking demonstrated on a boat race, and the clearest illustration you will find.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Reward hacking, and being surprised. The agent isn't cheating; it's doing precisely what the reward said.
- Tuning until it works in simulation, then meeting the reality gap.
- Choosing RL for a problem that was a bandit, and paying for the extra complexity in debugging.
At a glance
FieldFoundations
Core idealearn from reward through trial and error
Best forsequential decisions
Key riskreward gets gamed
Powersgame AI, robotics, RLHF
DifficultyIntermediate → Advanced
Flashcards for this concept
Question
—
Answer
—
1 / 4