Model Hub
A public repository of pretrained models anyone can download — the thing that democratised AI, and a software supply chain nobody is securing.
When not to use it
torch.loadon an untrusted pickle. It executes arbitrary code by design. That'scurl | bash.- Unpinned revisions. A model can be updated under a tag you tested.
- Trusting the licence field. It's a claim someone typed, and the audits say it's frequently wrong.
- Assuming your safety eval clears an unknown base. Sleeper Agents: backdoors survive the full stack.
Reach for something else instead
- Safetensors — same weights, cannot execute. Prefer it, always.
- A private mirror — for anything you depend on. Repos get deleted and gated.
- Training your own — expensive, and you know what's in it.
- A commercial API — someone else's supply chain, contractually.
Sources & further reading
- Wolf et al. (2020), Transformers: State-of-the-Art Natural Language Processing — the library and the standard interface that mattered more than any model.
- Hubinger et al. (2024), Sleeper Agents — a backdoor survives the full safety stack. This is a supply chain paper whether or not it says so.
- Longpre et al. (2023), The Data Provenance Initiative — licence tags are wrong at scale. Applies to model cards as much as datasets.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Loading pickles from strangers. It's remote code execution as a documented feature.
- Not pinning revisions, then debugging a change you didn't make.
- Reading a model's licence without checking what it descends from. Non-commercial ancestry doesn't wash out.
- Assuming a fine-tune removes a base model's problems. The evidence says it doesn't.
At a glance
FieldTools & Ecosystem
What it didturned reimplementing a paper into one line; the biggest democratising force in the field
What it isnpm for neural networks, at npm's 2012 security posture
The specific riskpickle executes code on load, by design
The mitigationsafetensors, pinned revisions, mirrors
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Model hub vs. package registry — same shape, same supply chain, same trust-by-default. One of them got exploited and then got security. The other hasn't been exploited yet.