A Closed-Loop Control Architecture for Reliable Constraint Satisfaction in LLM Text Generation
- Source
- arxiv.org
- Author
- Quan Zhou, Shahbaz Siddeeq, Mika Saari, Pekka Abrahamsson
- Date
Context
A text-generation study improved compliance with a calculated readability target by letting ordinary code decide whether an AI's edits were acceptable. The model wrote and revised text; a controller measured readability and rejected edits that removed source entities, numbers or keywords. This created a feedback loop with an explicit acceptance rule. Study abstract.
Across the reported experiments, single attempts met the target in 21.1–31.6% of cases, compared with 92.5–98.8% for the loop. Revisions averaged no more than two rounds. The result demonstrates control over the chosen formula: because that same formula guided revisions and measured success, it does not establish that people actually found the text easier to read. Results and scope.
- Single-prompt attempts hit a numeric target (like a word count or readability grade) only 21.1-31.6% of the time across four commercial LLMs, in the tests reported here.
- Wrapping generation in a 5-stage loop (generate, evaluate, adjust, archive, analyze) that gates every edit with deterministic code lifted target-hit rates to 92.5-98.8%, typically within two edit rounds.
- The controller also rejects any edit that drops source entities, numbers, or keywords, keeping content fidelity at 0.92-0.93 even while forcing the numeric target.
- The authors caution the result shows reproducible control over a declared, computable metric, not that the output is validated as easier for a human reader.
- Practical recipe: declare the acceptance condition as code, restrict the model to local edits, and gate every edit on a content check.
- LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
articleCompound Prompt Constraints in LLM Code Generation: A Factorial Study of Format, Persona, and UrgencyShrenik Jadhav, Nickalsa LaPlaca, Caleb Stone, Ashok Raja, Omar Ochoa, Vidhyashree Nagaraju
articleBeyond the Sycophancy Score: How Task, Model, and Pressure Shape LLM YieldingGuang Yang, Homa Hosseinmardi, Fengchen Liu, Amir Ghasemian
articleAn Empirical Evaluation of Cost-Efficient Large Language Models on Algorithmic Programming TasksChandimal Adikari, Nandika Herath
Checking sign-in…
Loading comments…