The missing condition was the text itself
Recraft traced its earlier model’s text failures to the training signal. Its image-caption pairs often described the scene while omitting the exact words in the image and their positions. The model learned common visual objects from repeated labels, but lacked an equally explicit condition for typography.
Text errors also carry an unusually high perceptual cost. Readers can overlook a small anatomical defect in an unfamiliar object, but a misspelled headline or broken label is immediately legible as failure. For design work, that makes typography more than a benchmark detail: one malformed word can disqualify an otherwise usable poster, package, or social graphic.
Recraft split typography planning from image rendering
The team added a text-layout image as a second conditioning signal alongside the caption, drawing on the approach described in TextDiffuser-2. Building that training data required a custom OCR system because the open-source recognizers Recraft tested did not perform well enough on its image distribution. Recraft then trained an image-captioning model conditioned on the OCR-derived layout, producing captions aligned with that layout.
At inference time, a language model predicts a compact text layout and the image generator consumes it in a ControlNet-like way. Users can also place the layout manually. That separation gives the renderer explicit words and positions instead of asking a single diffusion model to infer copy, ordering, placement, and appearance from a loose prompt all at once.
Why the pipeline matters
At inference time, Recraft separates layout prediction from final image rendering. OCR and layout-conditioned captioning are data-preparation stages, and Recraft says the layout generator and image generator can be trained independently. Recraft also says its original JSON layout format was ten times slower at inference than the compact format it ultimately used.
On Nov. 7, 2024, Recraft said V3 held first place with an Elo rating of 1172 on the Hugging Face Text-to-Image Model Leaderboard by Artificial Analysis. This is a dated vendor-reported result, not a claim about the current leaderboard. The post does not publish a text-specific accuracy score or an ablation isolating the layout signal. Its Elo result measures overall image preference, so it does not establish that this mechanism caused the ranking.
Explicit layouts add control, but introduce OCR, caption-alignment, word-ordering, and layout-generation failure points. The reusable hypothesis is to encode a repeatedly missed constraint explicitly, then test whether that representation improves the target failure.
