Legacy Bench Specialized Intelligence Index

Legacy software can produce plausible output while getting an essential business rule wrong. Factory's Legacy-Bench report, published September 24, 2026, describes a task in COBOL, an older business programming language, where a payroll program compiled and passed 24 of 29 tests, yet calculated a $275 health deduction instead of $125. The output looked plausible and the model reported success. Legacy-Bench, which Factory built, covers debugging, implementation, and migration tasks where exact calculations and record layouts matter. Factory's reading is that legacy environments give weak feedback and therefore need stronger verification; the example shows that a model's own success report is not reliable evidence of correct behavior on older systems.