OCR (Optical Character Recognition)
Turning pictures of text into text — solved for clean documents, still genuinely hard for everything else.
When not to use it
- When the text is already digital. Extracting from a PDF that has a text layer beats re-reading the pixels — check before you build.
- When layout is the point. Reading every character off an invoice without knowing which number is the total is not a result. That's document understanding.
- On low-resolution images, hoping. Below roughly 300 DPI, accuracy falls away and upscaling doesn't restore information that was never captured.
Reach for something else instead
- PDF text extraction when there's a text layer — exact, instant, free.
- Multimodal models when layout and meaning matter more than character-perfect transcription, and you can tolerate the cost and the hallucination risk.
- Structured data at source. Often the real answer is asking for a CSV rather than reading a picture of one.
Sources & further reading
- Graves et al. (2006), Connectionist Temporal Classification — the alignment trick underneath most text recognition.
- Shi, Bai & Yao (2015), An End-to-End Trainable Neural Network for Image-based Sequence Recognition — CRNN, the architecture most engines still resemble.
- Xu et al. (2020), LayoutLM: Pre-training of Text and Layout for Document Image Understanding — reading the page, not just the characters.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Testing on clean scans and deploying to phone photos. Perspective, shadow, and focus are a different problem entirely.
- Trusting a multimodal model's extracted numbers without validation. Unlike OCR, it fails silently with a plausible answer.
- Reporting character error rate as if it captures business impact. One wrong digit in a total barely moves CER and is a complete failure.
At a glance
FieldComputer Vision
Core ideapictures of text into text
Solved forclean printed scans
Hard forhandwriting, photos, layout
Floor~300 DPI
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
OCR vs. multimodal models — transcribing characters reliably vs. understanding a page and occasionally inventing it.