Meta-Learning: Learning to Learn Fast

Meta-learning trains a model to make use of a small set of examples for a new task. In Lilian Weng's 2018 explanation, training repeatedly presents miniature classification problems: a few labeled examples establish the categories, and separate examples test whether the model can use them.

This arrangement makes rapid adaptation part of the training objective. Success on one fixed collection of labels is no longer the whole task; the model must repeatedly work from limited evidence. Its ability to do that is learned across a family of training problems, so the small number of examples supplied for a new problem does not describe the total training effort.

“Humans, in contrast, learn new concepts and skills much faster and more efficiently.”

“This is why meta-learning is also known as learning to learn.”

“imo, the paper is poorly written, but the idea is still interesting.”

Lilian Weng

“It is worthy of mentioning that the Matching Networks paper refined the idea that training and testing conditions should match.”