Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls
Source
Dheeraj Mohandas Pai, Lu Xian
Author
Dheeraj Mohandas Pai, Lu Xian
Date
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → benchmarks confound instruction understanding with state tracking, so a long run that fails cannot be diagnosed. This setup gives per-step ground truth and shows how quickly excellent per-step accuracy compounds into end-to-end failure.
articleAgentic Security: A Systematization of Tools, Failure Modes, and Design Laws for LLM-Driven Penetration TestingIsrat Moyeen Noumi, Tarannum Ahmed Nowshin, Md. Mehedi Hasan Nipu, Mohammad Sakib Mahmood, Md. Jakir Hossain, M. F. Mridha
articleATLAS: Discovering Agent Strategies through LLM-Guided Abstraction and Automata LearningIgnacio D. Lopez-Miguel, Andreas Happe, J\"urgen Cito, Ezio Bartocci, Bettina K\"onighofer, Martin Tappler
articleBacktrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQsRuoxi Zhao, Maziar Raissi