The Tempe crash: it saw her for 5.6 seconds

TL;DR. On 18 March 2018 an automated test vehicle struck and killed Elaine Herzberg in Tempe, Arizona. The NTSB's final report found the system detected her about 5.6 seconds before impact and reclassified her repeatedly, as an unknown object, then a vehicle, then a bicycle, never correctly predicting her path. It could not classify a person as a pedestrian unless they were near a crosswalk. At 1.3 seconds it determined emergency braking was required, and emergency braking had been disabled in autonomous mode to reduce erratic behaviour. The system was not designed to alert the operator. The NTSB issued 19 findings, named the operator's distraction as probable cause and the company's safety culture as contributing. No criminal charges were brought against the company. The operator was prosecuted.

---

Status: established. Primary source: National Transportation Safety Board Highway Accident Report HAR-19/03, adopted 19 November 2019, with a preliminary report issued in May 2018. Figures are the NTSB's. Some secondary accounts give the speed as 39, 43 or 45 mph at different points in the sequence.

---

Elaine Herzberg was 49. She was crossing a four-lane road at night, pushing a bicycle, at a point with no marked crosswalk.

The vehicle was a Volvo XC90 modified by Uber's Advanced Technologies Group, operating in autonomous mode with a safety operator in the driver's seat as the testing permit required.

The sensors registered her about 5.6 seconds before impact. That is a long time. At the vehicle's speed it is roughly ninety metres of road.

What happened in those seconds is the finding. The system classified the object as unknown, then as a vehicle, then as a bicycle. Each reclassification discarded the tracking history and the predicted path with it. At no point did it correctly predict where she was going.

And it could not have identified her as a pedestrian. The NTSB found the system lacked the capability to classify an object as a pedestrian unless that object was near a crosswalk. She was not near one.

At 1.3 seconds before impact the system determined that emergency braking was required.

Emergency braking was disabled. Uber had switched off the vehicle's automatic emergency braking while under computer control, to reduce the potential for erratic vehicle behaviour. The system was also not designed to alert the operator that intervention was needed.

The operator was looking away. She began steering less than a second before impact and braked shortly after it.

The five and a half seconds, second by second

Reconstructed from the NTSB timings. The point of laying it out this way is that nothing in the sequence is a surprise to the system.

5.6 seconds. Radar and lidar return an object in the roadway. Classification: unknown. An unknown object has no predicted path, so no future position is assigned.

Roughly 5 to 3 seconds. Classification changes to vehicle. A vehicle in that position would be expected to move along the road, so the predicted path is generated on that assumption. It is wrong, and it is confident.

Roughly 3 to 1.5 seconds. Classification oscillates, including to bicycle. Each change discards the tracking history. A system that has watched something for four seconds has, from its own perspective, just seen it for the first time, repeatedly.

1.3 seconds. The system determines that emergency braking is required to mitigate a collision. This is the moment it becomes correct about the situation. Automatic emergency braking is disabled. No alert is sent to the operator.

Under 1 second. The operator, looking up, begins to steer.

Impact.

Four things stand out from the sequence and none is about sensing.

The system had four seconds of continuous returns and used none of them cumulatively. Persistence is information. Something that has occupied roughly the same region for four seconds is a thing, whatever label fits it, and a tracker that resets on reclassification throws that away.

The moment it became correct was the moment it could do nothing. At 1.3 seconds the correct action was identified and the mechanism to take it had been removed. The system's competence and its authority were decoupled.

There was no escalation path. Between "confident and wrong" and "certain and too late" there was no state in which the system could reduce speed, request attention, or otherwise act on the fact that it kept changing its mind. Repeated reclassification of an object in your path is itself a strong signal, and nothing consumed it.

And the human had no cue. The one component that might have intervened was given no information at any point in the sequence, because alerting had not been built.

A useful test for any system with a human backstop: at what point does the person find out? If the answer is that they are expected to notice on their own, the backstop is decorative.

The failure was categorisation, not perception

This is the part that transfers to systems with no wheels.

The sensors worked. Radar and lidar returned an object for five and a half seconds. Nothing was invisible and nothing was missed.

The system could not name what it was seeing, and its behaviour depended entirely on the name. A vehicle has an expected trajectory. A bicycle has another. An unknown object has none. Each time the label changed, the prediction restarted.

Which means the system was not uncertain in a way it could act on. It was confident, repeatedly, about mutually incompatible things. There is no record of it representing "I do not know what this is and it is in my path" as a state requiring caution, because caution was not a category. A classifier that must choose a label has no way to express that none of them fits.

And the pedestrian class had a precondition attached that the road did not honour. Building a classifier that only recognises pedestrians near crosswalks encodes an assumption about where people walk. People walk where they walk.

Every one of those is a design decision rather than a model deficiency. More training data would not have created an uncertainty state, retained tracking across reclassification, or removed the crosswalk precondition.

Everything protective had been switched off

The NTSB's findings list what was disabled in autonomous mode: automatic emergency braking, the driver alertness detection system, and road sign detection.

Each had a defensible reason. Emergency braking triggered by a system that misclassifies produces sudden stops for phantom obstacles, which is dangerous in its own right and makes a test programme unusable. The stated reason was to reduce erratic behaviour, and that is a real engineering concern.

Taken together they removed every layer that could have caught the classification failure. The system could not brake itself, could not tell the operator to, and had no independent check on its own attention. The design placed the entire safety margin on a human being asked to watch a road where nothing usually happens.

The NTSB named this directly. It found the company had not adequately recognised the risk of automation complacency or developed effective countermeasures, and that its inadequate safety culture contributed to the crash.

Automation complacency is not a character flaw. It is the predictable result of asking a person to supervise a system that is right almost all of the time. Vigilance decays because the task provides no feedback. This has been documented in aviation for decades, and it was designed around here as though it were a matter of instruction.

What a real incident report looks like

Article 118 in this series described public AI incident registries: mostly a title, a date and a link to a news story, with roughly 15% carrying any structured classification.

This report contains 19 findings, a stated probable cause, identified contributing factors, and formal safety recommendations to named recipients.

It establishes what the system detected and when, to a tenth of a second. It states which safety features were disabled and why. It reaches a conclusion about the operator's attention and about the company's safety culture, and it separates those rather than merging them. It says what would likely have happened had the operator been attentive.

That is possible because an independent investigator with subpoena power examined the vehicle's own data logs. No AI incident registry has that. No AI regulator currently has it either.

Which is the recurring finding of this record: the best documentation of AI failures comes from institutions built for something else. Securities disclosure produced the Zillow figures. A parliamentary inquiry produced the Dutch account. A transport safety board produced this. The AI-specific mechanisms produced the thinnest evidence in the entire series.

The accountability asymmetry

Worth stating precisely, because it is the most consequential outcome and it is not a technical one.

Prosecutors declined to bring criminal charges against the company. The safety operator was prosecuted.

The NTSB's probable cause was the operator's failure to monitor the road because she was visually distracted. That finding is in the report and it is not disputed here.

The same report also found that the company had not adequately managed the safety risk of its system's known limitations, had disabled the braking that might have mitigated the impact, had not designed any mechanism to alert the operator, and had an inadequate safety culture that contributed to her distraction.

Both sets of findings are in the same document. Only one produced a defendant.

That is not a claim about what the law should have done, which is outside what this record can establish. It is an observation about where liability lands when an automated system fails: on the person at the interface, who was placed there by a design decision they did not make.

What this establishes

Detection is not understanding. A system can track an object continuously for five seconds and still have no usable representation of what it is or where it is going.

A classifier with no uncertainty state cannot be cautious. If every input must receive a label, the system has no way to say that nothing fits, which is exactly the situation that warrants slowing down.

Class definitions encode assumptions about the world. A pedestrian class conditioned on crosswalk proximity is a statement about where people are allowed to be.

Disabling safeguards for smoothness is a cumulative decision. Each removal is individually reasonable and the aggregate leaves nothing.

And a human backstop is not a control unless it is designed as one. Placing a person in a supervisory role without alerting, without attention monitoring, and without a task that maintains engagement is not redundancy. It is an allocation of blame.

What is unresolved

Whether the classification instability was fixable at the time. Tracking continuity across reclassification is a known hard problem. Whether the state of the art in 2018 supported a better implementation is not something the report addresses.

Whether the crosswalk precondition was deliberate or emergent. The report states the capability was absent. It does not establish whether someone specified it or whether it fell out of how the training data was labelled.

How common the disabled-safeguard pattern was across the industry. Other programmes suspended testing after this crash, which suggests concern, and no comparable disclosure exists for any of them.

And whether the recommendations were adopted. The NTSB issues recommendations; it cannot compel. Tracking which were implemented, by whom, is not straightforward from public sources.

The counter-argument

The probable cause was operator distraction, and that finding should not be minimised. The NTSB concluded that an attentive operator would likely have had sufficient time to avoid or mitigate the crash. The system failed, and a person whose entire job was to catch that failure was looking away. Reframing the case as purely organisational understates a finding the investigators reached on the evidence.

Disabling emergency braking was not obviously wrong. A system that misclassifies objects and brakes hard on them is a hazard to everyone behind it and makes testing impossible. The decision was a trade-off between two real risks, and it is only clearly wrong in hindsight and only because of what the other layers failed to do.

Testing on public roads requires accepting some risk. No amount of simulation produces the situations that matter, and a policy of never testing until perfect is a policy of never deploying. The question is what risk is acceptable and who consents to bear it, and Elaine Herzberg did not consent to anything.

And the industry responded. Programmes were suspended, practices changed, and the report's findings on automation complacency became standard reference material. Whether that constitutes an adequate response to a death is a question this record cannot settle, and it is more than most of the other cases here produced.

The short version

On 18 March 2018 an automated test vehicle struck and killed Elaine Herzberg as she crossed a road in Tempe, Arizona, pushing a bicycle, at a point with no marked crosswalk.

The NTSB found the system detected her about 5.6 seconds before impact. It classified her as an unknown object, then a vehicle, then a bicycle, discarding its predicted path with each change and never predicting it correctly. It could not classify a person as a pedestrian unless they were near a crosswalk. At 1.3 seconds it determined emergency braking was needed, and emergency braking had been disabled in autonomous mode to reduce erratic behaviour, alongside driver alertness detection and road sign detection. The system was not designed to alert the operator.

The failure was categorisation, not perception. The sensors worked for five and a half seconds. The system was not uncertain in a way it could act on: it was confident, repeatedly, about incompatible things, because a classifier that must assign a label has no way to say none of them fits.

The report contains 19 findings, a probable cause, contributing factors and formal recommendations, which is possible because an independent investigator with subpoena power read the vehicle's own logs. No AI registry or regulator has that. Which continues this record's pattern: securities law documented Zillow, a parliamentary inquiry documented the Dutch case, a transport safety board documented this, and the AI-specific mechanisms produced the thinnest evidence in the series.

And the outcome. Probable cause was the operator's distraction. The same report found the company had not managed the known limitations of its system, had disabled the braking that might have mitigated the impact, had built no mechanism to alert the operator, and had a safety culture that contributed to her distraction. Both sets of findings are in one document. One produced a defendant.

Common questions

What happened in the Tempe self-driving crash? On 18 March 2018 a Volvo XC90 modified by Uber's Advanced Technologies Group, operating in autonomous mode with a safety operator aboard, struck and killed Elaine Herzberg as she crossed a four-lane road at night pushing a bicycle at a point with no marked crosswalk. The NTSB investigated and published its final report in November 2019.

Did the car see the pedestrian? Yes. The NTSB found the system registered radar and lidar observations about 5.6 seconds before impact. The failure was not detection but classification: it labelled the object unknown, then a vehicle, then a bicycle, discarding the predicted travel path with each reclassification and never predicting it correctly. It also lacked the capability to classify an object as a pedestrian unless that object was near a crosswalk.

Why didn't the car brake? At 1.3 seconds before impact the system determined an emergency braking manoeuvre was required. Uber had disabled the vehicle's automatic emergency braking while under computer control, to reduce the potential for erratic vehicle behaviour, relying instead on the human operator. The system was also not designed to alert the operator that intervention was needed.

What did the NTSB conclude? It issued 19 findings. Probable cause was the vehicle operator's failure to monitor the driving environment because she was visually distracted. Contributing factors included the company's inadequate management of the safety risks of its system's functional limitations, its design precluding emergency braking, its failure to recognise the risk of automation complacency and develop countermeasures, and an inadequate safety culture.

What is automation complacency? The decay of attention that follows from supervising a system that is correct almost all the time. It is not a character flaw but a predictable human response to a task offering no feedback, documented in aviation for decades. The NTSB found the company had not adequately recognised this risk or developed effective countermeasures, and that this contributed to the operator's extended distraction.

Who was held responsible? Prosecutors declined to bring criminal charges against the company. The safety operator was prosecuted. Both the finding about her distraction and the findings about the company's safety culture, disabled braking and absent alerting appear in the same NTSB report. Only one produced a defendant, which is an observation about where liability lands when automation fails rather than a claim about what the law should have done.

Why is this report significant beyond self-driving cars? Because it shows what an AI incident investigation can produce when an independent body with subpoena power examines a system's own data. Nineteen findings, a stated probable cause, separated contributing factors and formal recommendations, with timings to a tenth of a second. Public AI incident registries typically hold a title, a date and a link to a news story, and no AI regulator currently has comparable investigative power.

What is the transferable lesson for other AI systems? That detection is not understanding, and that a classifier required to assign a label has no way to express that none of them fits, which is exactly the situation warranting caution. Class definitions encode assumptions about the world, such as a pedestrian category conditioned on crosswalk proximity. Safeguards disabled individually for defensible reasons leave nothing in aggregate. And a human backstop without alerting, attention monitoring or an engaging task is not redundancy but an allocation of blame.