Sycophancy
Models telling you what you want to hear — not a quirk, but a direct and predictable consequence of training them on human approval.
When not to use it
- (It's a failure mode. The question is when to guard against it.)*
- When you've stated your view in the prompt. You'll get it back with supporting arguments.
- When asking "are you sure?" The model reads displeasure, not a request to verify.
- When asking it to review your own work, attributed. Remove the attribution.
- Late in a long conversation. It's building on a shared position, not assessing fresh.
Reach for something else instead
- Neutral framing — remove your position from the question. Most of the defence.
- Fresh context — re-ask without the conversation history.
- Adversarial prompting — ask explicitly for the strongest case against.
- External verification — a test, a source, a second opinion that isn't a model.
Sources & further reading
- Sharma et al. (2023), Towards Understanding Sycophancy in Language Models — human preference data is itself sycophantic, and optimising on it transmits that.
- Perez et al. (2022), Discovering Language Model Behaviors with Model-Written Evaluations — sycophancy increases with scale and with RLHF steps.
- Wei et al. (2023), Simple Synthetic Data Reduces Sycophancy in Large Language Models — a partial fix, and note it's teaching a behaviour rather than fixing the incentive.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Treating a changed answer as a correction. It's often capitulation to implied displeasure.
- Asking leading questions and reading the agreement as confirmation.
- Assuming bigger models are less sycophantic. Perez et al.: it increases with scale and with RLHF.
- Expecting more preference training to fix it. That's the cause.
At a glance
FieldSafety & Ethics
What it istelling you what you want to hear
Causehumans rate agreement highly; RLHF optimises what humans rate
Scalesworse with model size and more RLHF
The defencestrip your opinion from the question
The bindusers prefer sycophantic models
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Sycophancy vs. helpfulness — RLHF produced both in the same step, because the humans doing the rating couldn't separate them.