TPU
Google's chip built only for neural networks — the case that specialisation beats general-purpose hardware, and the only serious alternative to NVIDIA.
Reviewed July 14, 2026Stable
When not to use it
- With dynamic shapes. XLA needs static shapes and recompiles when they change. That'll dominate your runtime.
- With small batches. The array needs feeding. Underfed, you've bought nothing.
- When you need custom kernels. The CUDA ecosystem is where that flexibility lives.
- In PyTorch-native workflows. It works and it isn't the path of least resistance.
Reach for something else instead
- GPUs + CUDA — worse per watt, and the ecosystem is eighteen years deep.
- Inference-specific accelerators — serving is more standardised; the moat is thinner there.
- CPUs — for small models, still fine, and people forget.
Further reading
- Jouppi et al. (2017), In-Datacenter Performance Analysis of a Tensor Processing Unit — ISCA; unusually candid, including that it was memory-bound too.
- Kung (1982), Why Systolic Architectures? — the idea, thirty years before it mattered.
- Wang, Choi et al. (2019), bfloat16 and mixed-precision training — range beats precision, and the format everyone adopted.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Assuming specialisation escaped the memory wall. Jouppi's own paper says it didn't.
- Using small or variable batches. You've bought a systolic array and starved it.
- Thinking anyone can copy this. It exists because Google buys its own chips for its own workload.
- Missing that bfloat16 was the durable contribution. It's in every vendor's silicon now.
At a glance
FieldTools & Ecosystem
The questionif you only run neural networks, what's the chip?
The answera systolic array; weights stay put, data flows through, memory traffic mostly eliminated
Reported gains15–30× performance, 30–80× per watt vs. contemporary hardware
The honest findingit was memory-bound too
The lasting exportbfloat16
DifficultyAdvanced
Often compared with
TPU vs. GPU — one does the only operation that matters and does it beautifully; the other is worse per watt and has eighteen years of software nobody can rebuild.
Where this sits
A destination. 1 concept lead here, and nothing in the corpus depends on it.
1Levelsteps in
1Needs firstconcepts
0Opens upnothing further
1Areastays here
Learn these firstGPU
Computed from the prerequisite graph, not assigned. How this works