Red-teaming
Attacking your own system on purpose, before someone else does it for free.
Reviewed July 10, 2026Stable
When not to use it
- As a certificate. "We red-teamed it" describes an activity, not a property of the system.
- Only internally. The people who built it share its blind spots and are motivated not to find things.
- Without a remediation path. Findings that go into a document nobody owns are a record of what you knew and didn't fix.
Reach for something else instead
- Reduced scope — cheaper and more effective than testing whether a dangerous capability can be abused.
- Formal constraints — schema-bound output, permission limits, hard budgets. Testable properties beat adversarial hope.
- Staged rollout with monitoring — real users find things no red team imagined; the point is catching it early rather than at scale.
Read more on the blog
- AI alignment and safety, without the hype or dismissalAI safety is discussed either as impending doom or as overblown hype, and neither framing helps you understand it. The actual landscape: what alignment means, the concrete risks experts agree on, the speculative ones they don't, and why serious people land in very different places.
- What is AI jailbreaking? Why safety can be talked aroundAI models are trained to refuse harmful requests, yet people keep finding ways to make them comply anyway. Jailbreaking is the practice of talking a model around its own safety rules, and it has proven stubbornly hard to stop, because a model's safety is a thin behavioral layer laid over capabilities that were never removed.
- How to secure an LLM application: risks and defensesTraditional security rests on a boundary between instructions and data. A language model has no such boundary, because both arrive as one stream of text it cannot tell apart. That single fact determines every defense that works, and every one that does not.
Further reading
- Ganguli et al. (2022), Red Teaming Language Models to Reduce Harms — a large-scale effort described honestly, including what it missed.
- Perez et al. (2022), Red Teaming Language Models with Language Models — automating the attacker.
- Zou et al. (2023), Universal and Transferable Adversarial Attacks on Aligned Language Models — why manual red-teaming alone is now insufficient.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Counting findings instead of assessing severity, which rewards finding many harmless things.
- Red-teaming the model instead of the system. Incidents come from what it was wired to.
- Treating it as a launch gate rather than an ongoing practice. Capability changes reopen doors.
At a glance
FieldSafety & Ethics
Core ideaattack your own system deliberately
Best done bypeople who didn't build it
Outputa map of reachable harm, not a count
DifficultyIntermediate
Often compared with
Red-teaming vs. guardrails — searching for the holes vs. the walls you built.
Where this sits
A destination. 33 concepts lead here, and nothing in the corpus depends on it.
16Levelsteps in
33Needs firstconcepts
0Opens upnothing further
1Areastays here
Computed from the prerequisite graph, not assigned. How this works