Intel
Page 42
Curriculum for Reinforcement Learning
[Updated on 2020-02-03: mentioning PCG in the “Task-Specific Curriculum” section. [Updated on 2020-02-04: Add a new “curriculum through distillation&rdquo.
Why it mattersA structured deep-dive into how curriculum learning accelerates and stabilizes RL training — covering task ordering, procedural content generation, and distillation-based curricula.
articleLilian Weng
Scaling Laws for Neural Language Models
Established that language-model loss falls as a smooth power law in model size, data and compute — the result that turned scaling from a hunch into a budgeting exercise.
Why it mattersThis is where 'bigger reliably means better' got its evidence, and where the curve's shape — smooth and predictable across orders of magnitude — first let labs forecast a model's performance before training it.
articleJared Kaplan et al.
Self-Supervised Representation Learning
A survey of self-supervised representation learning, covering contrastive predictive coding and the momentum-contrast family including MoCo, SimCLR and BYOL.
Why it mattersA deep, well-organized survey of self-supervised representation learning methods (CPC, MoCo, SimCLR, BYOL) that gives engineers the conceptual grounding to build and choose embedding/pretraining approaches for AI-native products.
articleLilian Weng
Fast Transformer Decoding: One Write-Head is All You Need
Shares a single key-value head across all query heads, shrinking the KV cache that dominates memory during incremental decoding.
Why it mattersIt named the real bottleneck in generation: not the arithmetic, but the size of the key-value cache carried per token.
articleNoam Shazeer
Evolution Strategies
Stochastic gradient descent is a universal choice for optimizing deep learning models. However, it is not the only option.
Why it mattersExplains how evolution strategies can optimize objectives where gradients are unavailable or unreliable.
articleLilian Weng
Meta Reinforcement Learning
In my earlier post on meta-learning , the problem is mainly defined in the context of few-shot classification.
Why it mattersA clear, technical walkthrough of meta-RL methods for training agents that generalize to unseen tasks quickly — useful background for anyone building adaptive or few-shot-capable RL agents.
articleLilian Weng
Domain Randomization for Sim2Real Transfer
Deep RL is too sample-hungry to train on real robots, so models are trained in simulation and fail on the reality gap.
Why it mattersIf you're training RL policies in simulation and struggling to deploy them on physical robots, this breaks down how domain randomization over physical parameters (friction, mass, damping) closes the reality gap and where naive sim modeling fails.
articleLilian Weng
Are Deep Neural Networks Dramatically Overfitted?
[Updated on 2019-05-27.
Why it mattersA rigorous walkthrough of why overparameterized deep networks generalize instead of overfitting, covering the Lottery Ticket Hypothesis, intrinsic dimension, and generalization bounds.
articleLilian Weng
Meta-Learning: Learning to Learn Fast
[Updated on 2019-10-01: thanks to Tianhao, we have this post translated in Chinese !]
Why it mattersA rigorous, well-organized primer on meta-learning that walks through metric-based, model-based, and optimization-based approaches (including MAML and memory-augmented networks) with the math and intuition.
articleLilian Weng
Flow-based Deep Generative Models
GANs and VAEs never explicitly learn the density of real data because the integral is intractable.
Why it mattersA rigorous walkthrough of normalizing flows and models like RealNVP and Glow that explicitly learn tractable data likelihoods — useful for practitioners who need exact density estimation rather than the implicit distributions of GANs/VAEs.
articleLilian Weng
From Autoencoder to Beta-VAE
A tour from the plain autoencoder through denoising, sparse and contractive variants to the variational autoencoder and beta-VAE.
Why it mattersA single, rigorous walkthrough of the autoencoder-to-VAE lineage — including VQ-VAE, VQ-VAE-2, and TD-VAE.
articleLilian Weng
Attention? Attention!
[Updated on 2018-10-28: Add Pointer Network and the link to my implementation of Transformer.] [Updated on 2018-11-06.
Why it mattersA rigorous, implementation-backed walkthrough of attention and the Transformer architecture that grounds engineers in the mechanics behind modern LLMs, with code links for self-attention, Pointer Networks, and Neural Turing Machines.
articleLilian Weng
Implementing Deep Reinforcement Learning Models with Tensorflow + OpenAI Gym
A hands-on implementation walkthrough of deep reinforcement learning models in TensorFlow against OpenAI Gym.
Why it mattersBridges the gap between deep RL theory and working code, walking through actual TensorFlow + OpenAI Gym implementations of models most tutorials only describe abstractly.
articleLilian Weng
Policy Gradient Algorithms
A survey of policy gradient algorithms in reinforcement learning, from the basic theorem through actor-critic variants including SAC, D4PG, TD3 and SVPG.
Why it mattersA comprehensive, math-first survey of policy gradient methods that walks through the derivations and design tradeoffs of REINFORCE, A2C/A3C, DDPG, TD3, SAC, PPO, IMPALA and more in one place.
articleLilian Weng
A (Long) Peek into Reinforcement Learning
[Updated on 2020-09-03: Updated the algorithm of SARSA and Q-learning so that the difference is more pronounced. [Updated on 2021-09-19.
Why it mattersA rigorous, foundational walkthrough of core RL algorithms (SARSA, Q-learning, policy gradients) that grounds the concepts increasingly relevant to agent training and RLHF-style fine-tuning.
articleLilian Weng
The Multi-Armed Bandit Problem and Its Solutions
The exploration-exploitation dilemma stated as the multi-armed bandit problem.
Why it mattersA clear, code-backed walkthrough of bandit algorithms (epsilon-greedy, UCB, Thompson sampling) that maps the exploration/exploitation tradeoff to real problems like ad selection and A/B testing.
articleLilian Weng
Learning Word Embedding
Human vocabulary comes in free text.
Why it mattersA clear walkthrough of how free-text words become numeric vectors — from one-hot encoding to learned embeddings — grounding the intuition behind the embedding models and vector search that agentic engineers rely on daily.
articleLilian Weng
Anatomize Deep Learning with Information Theory
Naftali Tishby's information bottleneck applied to deep learning, using information theory to describe how a network's representations grow and transform over the course of training.
Why it mattersIt unpacks Tishby's Information Bottleneck framework and the two-phase (fitting then compression) view of DNN training, giving practitioners an information-theoretic lens on generalization that most engineering-focused writeups skip.
articleLilian Weng
From GAN to WGAN
A walk from the original generative adversarial network to Wasserstein GAN.
Why it mattersA rigorous, well-illustrated walkthrough of why vanilla GANs are unstable and how Wasserstein distance fixes the gradient/convergence problems — useful grounding for anyone building or debugging generative models.
articleLilian Weng
Attention Is All You Need
The 2017 paper that dropped recurrence and convolution for self-attention alone, introducing the Transformer — the architecture every large language model still builds on.
Why it mattersEvery model you use descends from this architecture, and reading it is how the rest of the stack stops being magic.
articleAshish Vaswani et al.
An index of the vibe-coding frontier. Corrections welcome.