Reality Is the Final Verifier: On Two Key Gaps in Agentic Software Engineering

Software can pass its tests and still disappoint its users because the tests depend on two imperfect descriptions: what people want and how the deployment environment behaves. This paper calls these the requirement gap and the model gap. Even a formal proof establishes correctness only against the requirements and assumptions it was given.

The authors propose using deployment evidence to revise those requirements, assumptions, or tests when actual behavior proves unacceptable. That changes the development loop: sometimes the specification or evaluator needs repair alongside the implementation. It is a framework for directing engineering effort toward failures that passing tests alone cannot settle.