Block Diffusion Interpolating Between Autoregressive And Diffusion Language Models 2025 03 12
Source
Cohere editorial sitemap
Author
Cohere editorial sitemap
Date
Terms in this piece · Glossary
KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
Why it matters
Diffusion LMs promise parallel, controllable generation but choke on fixed lengths and lose KV cacheThe memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.Full definition →. Block-wise interpolation with autoregressive modeling recovers both while keeping the parallelism.
articleBam Just Like That Simple And Efficient Parameter Upcycling For Mixture Of Experts 2024 08 15
articleMultilingual Arbitrage Optimizing Data Pools To Accelerate Multilingual Progress 2024 08 28
articleOne Tokenizer To Rule Them All Emergent Language Plasticity Via Multilingual Tokenizers 2025 05 30
articleInclude Evaluating Multilingual Language Understanding With Regional Knowledge 2024 11 29
