• fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.

If you're training or fine-tuning models with limited labeled data, this breaks down practical approaches to synthesizing training examples — augmentation techniques and few-shot LM-based generation — so you can bootstrap datasets without expensive labeling.

• postA super long overdue (3+ years?) post on scaling laws. Compute is expensive.
• articleFrom GAN to WGAN
• articleAnatomize Deep Learning with Information Theory
• articleLearning Word Embedding

• articleHow to Generate and Use Synthetic Data for FinetuningEugene Yan
• articleLearning with not Enough Data Part 1: Semi-Supervised LearningLilian Weng
• clipSynthetic data as a complement, at 13% of the pre-training mixAI Engineer

Checking sign-in…

Loading comments…