AI Alignment
The problem of making AI systems actually do what people intend — reliably pursuing the goals we want, not just the ones we accidentally specified.
When not to use it
- As a synonym for safety. Alignment is about systems pursuing intended goals; safety also covers misuse, reliability, security, and impact. Collapsing them hides real problems.
- As a reason to defer near-term duties. Long-term alignment debates don't excuse an unmonitored model making decisions about people today.
- As a marketing claim. "Aligned" is not a binary property a product can possess, and treating it as one is how the term gets emptied.
Reach for something else instead
- Evaluation and red-teaming — concrete, measurable, and what most teams actually need before they need alignment theory.
- Access control and scope limits. The strongest safety measure is usually not letting the system do the dangerous thing at all.
- Human oversight on consequential decisions, designed in rather than promised.
Sources & further reading
- Amodei et al. (2016), Concrete Problems in AI Safety — still the clearest framing of the near-term technical issues.
- Christiano et al. (2017), Deep Reinforcement Learning from Human Preferences — the technique behind RLHF.
- Bai et al. (2022), Constitutional AI — one approach to supervision that doesn't scale with human labellers.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Assuming a model that behaves well in testing is aligned. It's evidence about the test, not the system.
- Confusing refusing to say things with being aligned. A model can be harmless and still pursue the wrong objective.
- Treating this as purely technical. What "intended behaviour" means is a question about people, and it doesn't have a purely engineering answer.
At a glance
FieldSafety & Ethics
Core ideaAI doing what we intend, not just what we said
Failure modereward hacking
Current methodsRLHF, red-teaming, oversight
DifficultyIntermediate → Advanced
Flashcards for this concept
Question
—
Answer
—
1 / 4