Sebastian Raschka

A curated, categorized reading list of notable LLM research papers published from January to May 2026, organized into topics like architecture, reasoning, reinforcement learning, and agent systems. It's a reference resource for researchers and practitioners who want to track the field's developments.

An in-depth educational article that walks through the Qwen3 LLM architecture and reimplements it from scratch in pure PyTorch. It covers both the dense and Mixture-of-Experts variants, helping readers understand how modern open-weight models work under the hood.

An educational article that explains the four primary methods for evaluating large language models — multiple-choice benchmarks, verifiers, leaderboards, and LLM judges — with from-scratch PyTorch code examples. It helps readers build a mental map for interpreting benchmarks, leaderboards, and research papers.

A minimal but fully working coding agent implemented in pure Python, designed to demonstrate the six core building blocks of coding agents like Claude Code and Codex CLI. It serves as an educational reference for understanding how agent harnesses use tools, memory, and repo context to make LLMs more capable at software tasks.

An in-depth technical article explaining how large language models learn to support low-, medium-, and high-effort reasoning modes. It covers reasoning model training via RLVR, think tokens, on/off reasoning switches, and how reasoning effort settings work in models like GPT-5, Qwen3, and gpt-oss.

An in-depth educational article by Sebastian Raschka exploring alternative LLM architectures beyond standard autoregressive transformers, including linear attention hybrids, text diffusion models, code world models, and small recursive transformers. It breaks down architectures like Qwen3-Next, Kimi Linear, MiniMax-M1/M2, and DeepSeek V3.2 with technical explanations and code illustrations.

An educational article and visual gallery explaining the evolution of attention mechanisms in modern LLMs, from Multi-Head Attention (MHA) and Grouped-Query Attention (GQA) to Multi-Head Latent Attention (MLA) and sparse/hybrid architectures. Each concept is illustrated with visual model cards and real-world example architectures.

An in-depth technical article breaking down the architectural design choices in OpenAI's gpt-oss-120b and gpt-oss-20b open-weight models, comparing them against GPT-2 and Qwen3. It walks through MoE, RoPE, GQA, sliding-window attention, RMSNorm, MXFP4 optimization, and more.

A tutorial for assembling a fully local coding agent from open-source tools and open-weight models, pairing an agent harness with a model served by a local inference runtime.

An in-depth technical article walking through the architectural evolution of DeepSeek's open-weight LLMs from V3 to V3.2, explaining concepts like Multi-Head Latent Attention, RLVR reasoning training, and the new DeepSeek Sparse Attention mechanism.

A ~15-hour video course by Sebastian Raschka that teaches you how to build a large language model from scratch in Python — covering tokenization, attention mechanisms, pretraining, and finetuning. It serves as standalone companion content to his 'Build a Large Language Model (From Scratch)' book.

An in-depth educational article that categorizes and explains inference-time scaling techniques for improving LLM reasoning, covering methods like chain-of-thought prompting, self-consistency, best-of-N ranking, rejection sampling, self-refinement, and search over solution paths. It's a companion piece to a book chapter on building reasoning models from scratch.

An in-depth year-in-review article covering the major developments in large language models throughout 2025, including DeepSeek R1, RLVR, GRPO, inference-time scaling, and architectural trends, plus predictions for 2026 and 2027. Written for practitioners and researchers who want a technically grounded synthesis of the year's LLM progress.

A learning-oriented walkthrough of how to reverse-engineer and understand new open-weight LLM architectures by inspecting Hugging Face config files and reference implementations rather than relying on sparse technical papers. Useful for ML practitioners who want to deeply understand how modern model architectures actually work.

A hands-on technical tutorial that explains how key-value (KV) caches speed up LLM inference, walking through a from-scratch, human-readable PyTorch implementation. It compares text generation with and without caching and demonstrates a ~5x speedup on a 124M parameter model.

An in-depth technical article that compares the architectural designs of modern flagship open-weight LLMs, from DeepSeek-V3 and OLMo 2 to Kimi K2 and Gemma. It breaks down design choices like Multi-Head Latent Attention, Mixture-of-Experts, normalization placement, and QK-Norm.

A topic-organized collection of 200+ large language model research papers from the first half of 2025, curated by Sebastian Raschka. Papers are grouped by themes like reasoning models, reinforcement learning, and multimodal models for easy reference.

A technical roundup comparing ten open-weight LLM architectures released in early 2026, including Arcee Trinity, Kimi K2.5, Step 3.5 Flash, Qwen3-Coder-Next, and GLM-5. It breaks down architectural details like Mixture-of-Experts, sliding window attention, gated attention, and multi-token prediction for each model.

An in-depth technical article analyzing recent open-weight LLM architecture innovations focused on long-context efficiency, covering KV sharing and per-layer embeddings in Gemma 4, attention budgeting in Laguna XS.2, compressed convolutional attention in ZAYA1-8B, and DeepSeek V4's mHC design. Useful for ML engineers and researchers who want to understand how modern transformers reduce KV cache size and attention costs.