Structured Output
Making a model return JSON that always parses — solved at the format layer, and still wide open at the correctness layer.
Reviewed July 11, 2026Stable
When not to use it
- When a human reads the output. Prose is the right format for people. Don't schema your way into worse writing.
- When the task needs reasoning and you're constraining from the first token. Let it think in text, then structure in a second pass.
- When the shape is genuinely unknown. Forcing a schema onto something that doesn't have one produces confident nonsense in well-formed fields.
- As a substitute for validation. Schema-valid is not business-valid.
{"age": 900}parses fine.
Reach for something else instead
- Function calling — when you want an action rather than a value.
- Free text plus a parser — fine when the format is trivial and failures are cheap.
- Two-pass: reason then structure — more reliable on hard tasks, costs an extra call.
- Deterministic extraction — regex or a parser, when the input is actually regular. Faster and exact.
Read more on the blog
- Prompt engineering in 2026: what still works and what changedIf your prompts still open with "act as an expert" and "let's think step by step," you're using 2023 advice on 2026 models, and some of it now makes your output worse. What actually moves results today, what quietly stopped working, and why.
- How to reduce AI hallucinations: what actually worksYou cannot instruct a model into being truthful, because the process that invents a fact is the same one that recalls a real one. Every technique that measurably reduces hallucination works by adding something outside the model. Here is what works, in order of impact, and what only appears to.
Further reading
- Willard & Louf (2023), Efficient Guided Generation for Large Language Models — the finite-state machine approach behind most constrained decoding.
- Geng et al. (2023), Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning — grammar-constrained generation, and its costs.
- Tam et al. (2024), Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models — the evidence that constraint can degrade reasoning.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Believing valid means correct. It means parseable. Those are different bugs and only one of them wakes you up.
- Cryptic field names. The names are prompts;
xandcustomer_sentimentproduce different answers. - No
"unknown"or"other"option, which forces a fabrication and stores it as fact. - Constraining a reasoning-heavy task from the first token and losing quality to the grammar.
- Skipping business validation because the schema passed. Types are not semantics.
At a glance
FieldAI Agents
What's guaranteedthe format
What isn'tthe content
Best mechanismconstrained decoding
Schema designfield names are prompts
Hidden costconstraint can degrade reasoning
DifficultyIntermediate
Often compared with
Structured output vs. "reply in JSON please" — one is enforced at the sampler, the other is a request. The second works most of the time, which is the worst possible reliability.
Where this sits
A destination. 27 concepts lead here, and nothing in the corpus depends on it.
14Levelsteps in
27Needs firstconcepts
0Opens upnothing further
1Areastays here
Learn these firstFunction Calling
Computed from the prerequisite graph, not assigned. How this works