Speech Emotion Recognition
Detecting how someone feels from their voice — deployed at scale in call centres, and the psychology says the thing it measures may not exist.
When not to use it
- To claim someone's internal state. The evidence doesn't support reading emotion from expression reliably. That's the whole finding.
- In workplaces or education, in the EU. Prohibited, with narrow exceptions, because a regulator read the evidence.
- Trained on acted data, deployed on natural speech. Acted anger is a performance of a stereotype — the exact thing that doesn't generalise.
- Across cultures without validation. Vocal norms vary enormously. An animated baseline reads as angry.
Reach for something else instead
- Arousal only — genuinely detectable; it has physiological correlates. Say that's what you measured.
- Perceived emotion, labelled as perceived — answerable, useful, honest.
- Asking the person — unglamorous, and it's the only direct measurement available.
- Not doing it — what the EU concluded for workplaces.
Sources & further reading
- Barrett et al. (2019), Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements — the evidence review. Read this before building or buying anything here.
- Busso et al. (2008), IEMOCAP: Interactive emotional dyadic motion capture database — the standard benchmark, and it's acted.
- Stark & Hoey (2021), The Ethics of Emotion in Artificial Intelligence Systems — what's being claimed versus what's supported.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Treating benchmark accuracy as validation. You can score well on actors performing stereotypes for annotators guessing.
- Conflating arousal with emotion. High arousal is anger, excitement, urgency, or a bad line.
- Ignoring inter-annotator agreement. It's low, and it's telling you the task isn't defined.
- Assuming the basic-emotions premise is settled science. It's a 1960s hypothesis the evidence doesn't support.
At a glance
FieldSpeech & Audio
The premiseemotions have characteristic expressions you can read off
The evidencedoesn't support it; people scowl when angry under 30% of the time (Barrett et al., 2019)
What is measurablearousal, which has physiological correlates
What isn'tdiscrete emotion categories
Statusprohibited in EU workplaces and schools
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Arousal vs. emotion — one has physiological correlates and is measurable from voice; the other is a category the evidence says isn't reliably expressed. Systems report the second and measure the first.