Vibeleaderboard
← All Intel
Intel / article

Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters

Source
arxiv.org
Author
Charlie Snell et al.
Date

Context

Extra computation while answering works better when its strategy matches the problem. In this 2024 study, researchers tested specially fine-tuned models on mathematics problems, comparing repeated answer revision with searching solutions using a model that scores intermediate steps. Easier problems could benefit from refining an existing answer; harder ones could require exploring different approaches. Choosing the strategy according to difficulty improved efficiency over generating independent answers and selecting the highest-scoring one. The paper reports more than fourfold compute savings in its comparisons. The result concerns allocating work among tested methods, rather than merely asking a model to produce a longer response.

Terms in this piece · Glossary
  • inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Recommended reads
Comments

Checking sign-in…

Loading comments…