• Gemma 4 E2B has 35 layers but only the first 15 compute their own key/value projections; the last 20 reuse KV from the nearest earlier layer of the same attention type. That roughly halves the cache, about 2.7 GB saved at a 128K context.
• Layers that share KV still compute their own queries, so each one forms its own attention pattern; only the memory-heavy keys and values are recycled. The cost is reduced capacity, which the cross-layer attention paper found minimal at small scale.
• Per-layer embeddings explain why Gemma 4 E2B is quoted as both 2.3B and 5.1B parameters. The extra weight sits in embedding tables looked up once per token and injected as a small per-layer slice, so the transformer stack stays as cheap as a 2B model.
• The reason architecture work swung toward KV cache and memory traffic: reasoning models and agents keep far more tokens around for far longer, so long-context memory rather than raw compute became the binding constraint.

• open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
• context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
• KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
• transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.

If you're building or optimizing long-context LLM inference, this breaks down how the newest open-weight models actually shrink KV cache and attention costs — with comparative coverage of Gemma 4, DeepSeek V4, ZAYA1, and Laguna plus from-scratch code you can adapt.

• articleA Dream of Spring for Open-Weight LLMs: 10 Architectures from Jan-Feb 2026
• articleBeyond Standard LLMs
• articleFrom GPT-2 to gpt-oss: Analyzing the Architectural Advances
• articleUnderstanding the 4 Main Approaches to LLM Evaluation (From Scratch)

• articleThe Big LLM Architecture ComparisonSebastian Raschka, PhD
• articleA Visual Guide to Attention Variants in Modern LLMsSebastian Raschka, PhD
• articleBeyond Standard LLMsSebastian Raschka, PhD

Checking sign-in…

Loading comments…