Vibeleaderboard
← All Intel
Intel / article

tufazip: Learning to Compress a Billion Bytes of Wikipedia

Source
tufalabs.ai
Date
Why it matters

Shows a single , trained online and seeded from a 4.33 MB offline prior, can approach hand-mixed compressors on 1 GB of Wikipedia. Ablations on replay, vocabulary size and model size give practitioners concrete tradeoffs for compression-as-.

Key takeaways · AI-distilled
  • tufazip compressed the billion-byte enwik9 Wikipedia file to 105.39 MB including its decoder and offline weights, taking #2 on Matt Mahoney's Large Text Compression at submission (Sept. 7, 2026) and the best entry using a neural network as its only predictor.
  • The encoder and decoder start from the same weights and make the same deterministic training update after every , so the decoder rebuilds the model's online learning as it decodes and the weights learned online never have to be stored in the archive.
  • Replay, extra training on already-decoded text, helped consistently but costs compute: widening the window from 15M to 35M tokens cut the enwik9 payload from 107.22 to 105.92 MB (1.21%) for about 40% more estimated model FLOPs.
  • A 4K word vocabulary beat 8K and 16K, and dedicated XML/wiki tokens made compression worse despite the heavy markup. More parameters did not reliably help: on enwik9 a 267M model produced slightly larger payloads than a 188M one.
  • An offline prior must save more payload than its own size. Two shared blocks stored once (4.33 MB) and copied into 12 layers netted 0.59 MB over pure online learning, while a six-layer prior predicted better but its 10.9 MB decoder-and-weights bundle made the total larger.
Terms in this piece · Glossary
  • transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
  • token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
Read the source tufalabs.ai
Recommended reads
Comments

Checking sign-in…

Loading comments…