Field
Tools & Ecosystem
What you run models on, and what it costs.
The practical layer. Not what models are — what it takes to actually use one.
The central decision is whether to rent or own. An inference API means someone else buys the hardware and keeps it running, and the arithmetic favours that for far longer than most teams assume. Open-weight models let you download and run them yourself — genuinely valuable when data can't leave or you need to work offline, and usually not "open source" despite the label: weights without training data is closer to a compiled binary.
GPUs are what everything runs on, and the specs people shop for are the wrong ones — memory decides what runs at all, bandwidth usually decides speed, and raw compute is rarely the bottleneck. Quantization is how large models fit on hardware you own, at a cost that isn't spread evenly. Vector databases are the storage under semantic search.
Start with Inference API — it's how nearly everyone actually uses AI.
16 concepts in this field
Vector Database
A database built to store embeddings and find the most similar ones fast — the search engine behind meaning-based retrieval.
Open-Weight Models
Models whose weights you can download and run yourself — often called open source, usually not quite.
GPU
The chip that made deep learning possible — thousands of small cores doing the same maths at once, which is exactly what neural networks need.
Inference API
Renting a model by the request — how nearly everyone actually uses AI, and the dependency that comes with it.
Semantic Search
Searching by meaning rather than by words — which finds what keyword search misses, and misses what keyword search finds.
Knowledge Graph
Facts as a network of entities and relationships — symbolic AI's one commercial survivor, and RAG is rediscovering it.
MLOps
The engineering around a model that makes it a system rather than a notebook — and the model is a few percent of it.
Model Serving
Getting a trained model to answer requests reliably — where the model is the easy part and the queue is the hard one.
Batching
Processing many requests together to use hardware that's mostly idle — the largest cost lever in inference, and it costs you latency.
Edge AI
Running models on the device instead of a server — for privacy, latency and cost, against a memory wall that doesn't move.
Federated Learning
Training a shared model across devices without collecting the data — and "your data never leaves" is not the guarantee it sounds like.
Vector Search
Finding the nearest vectors to a query, fast — by not actually finding them, which almost nobody measures.
CUDA
NVIDIA's platform for programming GPUs — and the actual reason NVIDIA has no competition, which is not the silicon.
TPU
Google's chip built only for neural networks — the case that specialisation beats general-purpose hardware, and the only serious alternative to NVIDIA.
PyTorch
The framework that won by being easier to debug — a lesson about developer experience that the industry keeps having to relearn.
Model Hub
A public repository of pretrained models anyone can download — the thing that democratised AI, and a software supply chain nobody is securing.