Garry Tan wants US open-weight AI labs to 'distill' frontier models, too
- Source
- techcrunch.com
- Author
- TheJCDenton
- Date
Distillation trains a smaller, cheaper model to imitate a larger frontier model's outputs, letting a team without frontier-scale compute reach much of that capability at a fraction of the cost. Anthropic has pushed regulators to restrict the practice when it is used by Chinese labs to catch up on frontier capability without the underlying research cost. Garry Tan's pushback reframes that as a competitiveness problem rather than a security one: if distillation restrictions apply broadly, US open-weight labs lose the same shortcut, while closed frontier labs keep their advantage. His position is that the US should let its own open-weight ecosystem distill frontier models too, rather than writing rules that end up protecting incumbents more than they slow any particular competitor. The outcome of this argument matters directly for smaller labs and open source projects building on top of frontier APIs today. Rules written to stop distillation abroad rarely stop at the border, and how regulators eventually define and police the practice will shape what a startup with a few GPUs and API access is legally allowed to train on.
Context
Y Combinator CEO Garry Tan has argued, in an interview reported by TechCrunch, that American labs should be able to distill American frontier models. He says there "should be an American regime" and calls restrictions on what customers do with outputs from closed models constraining.
Distillation is a training method in which one model produces answers to a set of inputs and a smaller model learns to imitate them. In its September report, Anthropic calls distillation itself a legitimate training method, and defines the illicit form as a covert, large-scale effort to extract a model's capabilities without permission, typically enabled by fraud such as fake accounts. The dispute Tan raises is therefore about who may use outputs from a provider's model to build competing systems, and under what access terms, rather than about the technique.
- open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
- distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
Checking sign-in…
Loading comments…