Intel / article

Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation

Source: arxiv.org
Author: Atul Anand, Sourav Chattaraj
Date: Published Aug 5, 2026

Context

Adding requirements to a prompt can introduce contradictions as well as make the task harder. An instruction-following study combined up to twenty constraints and found that some could not be satisfied together. Its JSON-output requirement, for example, conflicted with nine other instruction types. The reported decline in compliance therefore includes incompatible demands, not simply a model forgetting instructions. Study abstract.

The researchers also tested an instruction compiler: a model rewrites the requirements once, and subsequent queries reuse that prompt. They report improvements of up to eleven percentage points for weaker models, with stronger models essentially unchanged. Rewriting helped in some settings, but the benefit depended on the model. Reported results.

Key takeaways · AI-distilled

Stacking verifier-checked instructions (1 to 20 at a time, from a pool of 24) degrades follow rate non-linearly: from ~96% down to as low as 20% across Claude Sonnet 4.6, GPT-5-mini and Gemini 2.5 Flash.
The damage is structured, not random: reproducible pairwise conflicts drive the collapse - a single output-JSON constraint is jointly unsatisfiable with nine of the other instructions.
A training-free instruction compiler (one call that rewrites the stacked prompt, reused across queries) recovers up to +11 points of follow rate - but only for weaker models; stronger models already internalise the structure.
Controls attribute the gain to the rewrite itself, not extra , reordering or measurement headroom - so prompt compilation is worth it exactly where cheap models are deployed at scale.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Key quotes

“Instruction-following degrades non-linearly: the follow rate falls from ~96% to as low as 20%, driven by a structured and reproducible set of pairwise conflicts.”

“A single "output JSON" constraint, for example, is jointly unsatisfiable with nine others.”

“It recovers up to +11 points of follow rate for weaker models, which are also the models most often deployed at scale, while leaving stronger models, which already internalise the same structure, essentially unchanged.”

Read the source arxiv.org