Loops, not one-shots: how coding agents got a tokenizer trainer to production

Liquid AI's coding-agent experiment found that passing small tests did not establish whether generated software could handle production data. Two agents produced working toy versions of a tokenizer trainer, software that learns how to divide text into model inputs. Both initially failed at full scale. Experiment account.

Repeated runs exposed problems such as excessive memory use and incorrect vocabulary ordering. The agents received those failures and revised their implementations, while two external libraries checked outputs through tests the agents could not change. One development track eventually completed the production workload; the other was stopped. The account supports a specific engineering lesson: realistic workloads and independent checks can reveal failures that an agent's own small tests miss.