Counterexamples as Feedback for Agent Self-Correction

Showing an AI exactly where its answer fails can make correction more effective than asking it to try again. A-CEGIS tests this with regular expressions, patterns that decide whether text matches a rule. A checker gives the agent examples it wrongly accepted or rejected, which guide its next revision. On 30 tasks, the authors report 90% success within four turns with diagnostic feedback, versus 27% with generic self-correction and 17% on the initial attempt. This small experiment supports evaluating agents by how well they use concrete feedback, alongside whether their first answer is correct. Study abstract.