Lilian Weng

Lilian Weng announces a long-planned post on scaling laws: how they let you reason about the optimal split of compute between data and model size before committing to an expensive run.

Most task-specific labelled data comes from human annotation, which makes annotator disagreement, expertise and incentives a modelling problem rather than an operational detail. On what high-quality human data actually requires.

A rewritten and roughly doubled survey of transformer architecture variants, restructured and brought up to date with the improvements proposed since the original 2020 version.

When facing a limited amount of labeled data for supervised learning tasks, four approaches are commonly discussed.

The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to each other while dissimilar ones are far apart. Contrastive learning can be applied to both supervised and unsupervised settings. When working with unsupervised data, contrastive learning is one of the most powerful approaches in self-supervised learning .

Approaches to generating synthetic training data when real data is scarce, covering augmentation of existing samples and outright generation of new ones.

[Updated on 2022-03-13: add expert choice routing .] [Updated on 2022-06-10]: Greg and I wrote a shorted and upgraded version of this post, published on OpenAI Blog: “Techniques for Training Large Neural Networks”

This is part 2 of what to do when facing a limited amount of labeled data for supervised learning tasks. This time we will get some amount of human labeling work involved, but within a budget limit, and therefore we need to be smart when selecting which samples to label.

Reward hacking is an agent exploiting flaws in a reward function to score well without doing the task, and it persists because specifying rewards accurately is fundamentally hard, a problem RLHF inherits at scale.

[Updated on 2020-09-03: Updated the algorithm of SARSA and Q-learning so that the difference is more pronounced. [Updated on 2021-09-19: Thanks to 爱吃猫的鱼, we have this post in Chinese ].

A survey of self-supervised representation learning, covering contrastive predictive coding and the momentum-contrast family including MoCo, SimCLR and BYOL.

A survey of prompt engineering as in-context steering of an autoregressive model without touching its weights, treated as an empirical science whose methods vary sharply between models.

[Updated on 2019-05-27: add the section on Lottery Ticket Hypothesis.] If you are like me, entering into the field of deep learning with experience in traditional machine learning, you may often ponder over this question: Since a typical deep neural network has so many parameters and training error can easily be perfect, it should surely suffer from substantial overfitting. How could it be ever generalized to out-of-sample data points?

Stochastic gradient descent is a universal choice for optimizing deep learning models. However, it is not the only option. With black-box optimization algorithms, you can evaluate a target function $f(x): \mathbb{R}^n \to \mathbb{R}$, even when you don’t know the precise analytic form of $f(x)$ and thus cannot compute gradients or the Hessian matrix. Examples of black-box optimization methods include Simulated Annealing , Hill Climbing and Nelder-Mead method .

A tour from the plain autoencoder through denoising, sparse and contractive variants to the variational autoencoder and beta-VAE, with the ELBO loss and the reparameterization trick worked through, plus VQ-VAE and TD-VAE.

Why inference on large transformers is hard and what actually helps: the memory and latency bottlenecks that make powerful models expensive to serve, and the optimisation techniques including distillation that address them.

GANs and VAEs never explicitly learn the density of real data because the integral is intractable. Flow-based models do, by composing invertible transformations, and this is how they work.

A survey of LLM-powered autonomous agents built around planning, memory and tool use, drawing on AutoGPT, GPT-Engineer and BabyAGI as early proofs of concept.

A hands-on implementation walkthrough of deep reinforcement learning models in TensorFlow against OpenAI Gym, covering naive Q-learning, DQN, double and duelling variants, policy gradients and actor-critic.

Naftali Tishby's information bottleneck applied to deep learning, using information theory to describe how a network's representations grow and transform over the course of training.

[Updated on 2019-10-01: thanks to Tianhao, we have this post translated in Chinese !]

Large pretrained language models are trained over a sizable collection of online data. They unavoidably acquire certain toxic behavior and biases from the Internet. Pretrained language models are very powerful and have shown great success in many NLP tasks. However, to safely deploy them for practical real-world applications demands a strong safety control over the model generation process.

A survey of adversarial attacks and jailbreak prompts against language models, and why alignment work such as RLHF builds default-safe behaviour that these techniques are designed to route around.

Diffusion models moved from image synthesis to video, a strictly harder problem because temporal consistency across frames demands world knowledge a single image never needed.