A decent text-to-image model trained for ~$20K (Apache 2.0 Weights)

Linum is a two-person lab, run by the brothers Sahil and Manu Chopra, that trains its own video-generation models from scratch. In a field notes post it released a text-to-image model with about 2 billion parameters as open weights under the Apache 2.0 license. The price in the title is a compute bill: $21,038 in total, made up of $20,348 for pretraining (8,139 H100-hours on 64 H100 GPUs, restarts included) and $690 for post-training (276 H100-hours), all priced at $2.50 per H100-hour. The founders' Hacker News post rounds that to about $20K. The figure is the cost of this one run. The post gives no cost for the earlier experiments that produced the design.

The model is a test of an architecture rather than a product. Linum is building toward a video model, Linum v3, whose main goal is cheaper generation, and the post explains that video models almost always start as image models: they learn people, places and things before they learn movement. Linum names attention as the cost bottleneck for video. Attention compares every token of the input with every other, so its cost grows with the square of the input length. Most image generators shorten the input with a separate compression model, a variational autoencoder or VAE, and the diffusion model then learns to generate inside that compressed representation. Linum's Pyramid-JiT design drops the VAE and works directly on pixels, cutting each image into patches of 32 by 32 pixels, so each token stands for a square of the image twice as wide as in most models, where a token covers 16 by 16 pixels. That lets it represent a 512-pixel-square image with as many tokens as Linum's previous model, Linum v2, used for a 256-pixel-square one, which Linum describes as a 4× reduction in the attention context window.

In an earlier post, Linum reported that this design converged in 11 times fewer training samples than Linum v2, a fairly standard VAE-based model, but those experiments ran for 50 to 100 million samples. This run checks whether the design holds up over much longer training: 610 million samples at 512 pixels, then 50 million at 1024 pixels, followed by a short post-training stage using a variant of a method called Finite Difference Flow Optimization with JEV-Image and human reviewers as judges. Prompts are read by a frozen Qwen3.5-4B language model from Qwen, which is not trained further.

The evidence for quality is Linum's own. The founders say they were surprised by how good the results looked for a model this size, and the post shows a gallery of 249 samples, but its text reports only Linum's own validation loss, not a benchmark comparison with other image models. This explanation is based on the post's text; the gallery images and charts were not examined. The post is direct about limits. The model was trained without images of rendered text, which Linum filtered out as irrelevant to animation, so it cannot write text or make posters. It also struggles with full-body figures, scenes with many entities such as six cars or crowds, interactions between several people, and world knowledge, and Linum says a larger model, for example 7 billion parameters, and further post-training would likely help. The run shows that a two-person lab could train a 2-billion-parameter image model on its own pixel-space architecture for about $21K of compute and judge it good enough to release. The post does not show whether the approach makes video generation cheaper; this model is Linum's stepping stone toward that.