• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
• test-time compute — Spending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.

Inference-time scaling results are almost always reported in English. This measures where extra samples and better selection actually buy accuracy in other languages, which changes how you budget compute for non-English traffic.

• articleCommit0 Library Generation From Scratch 2024 12 02
• articleLlmcrit Teaching Large Language Models To Use Criteria 2024 03 02
• articleSeacrowd A Multilingual Multimodal Data Hub And Benchmark Suite For Southeast Asian Languages 2024 06 14
• articleFishing For Magikarp Automatically Detecting Under Trained Tokens In Large Language Models 2024 05 08

• articleScaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model ParametersCharlie Snell et al.
• articleCategories of Inference-Time Scaling for Improved LLM ReasoningSebastian Raschka, PhD
• articleAutoscaling Endpoints for LLM InferenceTogether AI

Checking sign-in…

Loading comments…