PyTorch
The framework that won by being easier to debug — a lesson about developer experience that the industry keeps having to relearn.
When not to use it
- Optimising before profiling. The bottleneck is essentially never where you think.
- Without
torch.compile, on a hot path. It's one line for fused kernels. .item()or.cpu()in a training loop. Each one is a synchronisation point that stalls the GPU.- Assuming eager mode is fast. It's flexible. Flexible and fast are the thing
compileexists to reconcile.
Reach for something else instead
- JAX — functional, compiled, excellent for research at scale. The serious alternative.
- TensorFlow — still fine, still deployed, lost the ecosystem.
- Higher-level wrappers — Lightning and friends, for the parts that are boilerplate.
- ONNX — for exporting a model to somewhere that isn't PyTorch.
Sources & further reading
- Paszke et al. (2019), PyTorch: An Imperative Style, High-Performance Deep Learning Library — the design argument, made explicitly.
- Abadi et al. (2016), TensorFlow: A System for Large-Scale Machine Learning — the case for static graphs. It's a good case.
- Ansel et al. (2024), PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation — how the reconciliation works.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Believing the better-engineered system wins. Static graphs were sound engineering and lost to a debugger that worked.
- Syncing in a loop.
.item()stalls the GPU and it's the most common performance bug. - Skipping
torch.compile. One line, fused kernels. - Reading the framework war as trivia. Developer experience compounds into ecosystem, which is the moat.
At a glance
FieldTools & Ecosystem
How it wondefine-by-run; your code executes when you run it, so
print and pdb workWhat it beata better-resourced, better-engineered static-graph system with a superior production story
The reconciliation
torch.compile; flexibility by default, most of the performance in one lineThe lessonergonomics compound into ecosystem
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Define-by-run vs. define-and-run — one runs your code and lets you debug it; the other builds a graph it can optimise and you can't inspect. The debuggable one won and then borrowed the compiler.