Artifact Distillation: Training-Free Transfer from Frontier to Small Language Models
Source
arxiv.org
Author
Mustafa Safa Ozdayi, Somin Wadhwa
Date
Why it matters
A stronger model can write a reusable procedure bank that a cheaper model queries at inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition →, improving cheap-model quality without fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition →. Evidence is limited to four tasks.
Terms in this piece · Glossary
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
articleBroken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch ThemAditi Patodiya
articleNot All Tokens Are Equal: Inflation-Aware Routing for Agentic LLM SystemsHeming Fu, Shan Lin, Qianqian Xie, Guojun Xiong
articleForgeTrain: Forging Production-Grade Training Frameworks via Harness-Driven AI DevelopmentQingfeng He, Zhui Zhu, Shangzhan Li, Yaojian Chen, Haojun Sun, Xu Chen, Leshan Li, Yifei Shen, Changjingxing Zhao, Mengyuan Fan, Wenyu Guan, Yiyun Zheng, Yuxuan Zuo, Zhen Li, Zhenghang Luo, Yuxuan Li, Xu Han, Zhiyuan Liu