Pass@1 misses how well a model uses feedback after failures, which is how agents work. Seq@K shows model rankings can change under sequential feedback, so benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → choice may alter which model you pick.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
articleFinding the Move Is Not Winning the Game: XiangqiBench for Closed-Loop Evaluation of LLM AgentsYekun Chai, Qiwei Peng, Haoyi Xiong
articleEvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language ModelsXinning Li, Kemunto Ochwang'i, Aryasomayajula Ram Bharadwaj, Alexandra Souly, Robert Kirk
articleEvaluating Model Capabilities After Benchmark SaturationMorio Hamada, Jehyeok Yeon, Matteo Migliarini, Justin Olive, Magda Dubois