M31.4 CONNECT THE MECHANISM
Distinguish examples in a prompt from learning to adapt weights
Three recordings of a bird call you've never heard. Is that enough? Meet three ways to learn from a handful of examples, and find out which ones change the model's weights.
LESSON OVERVIEW15 min lesson
Lesson overview
Three recordings of a bird call you've never heard. Is that enough? Meet three ways to learn from a handful of examples, and find out which ones change the model's weights.
What you’ll explore
- Set up an N-way K-shot episode, classify with prototypes, explain MAML's inner and outer loops, and tell in-context learning (no weight change) apart from fine-tuning and meta-learning.
GO TO THE SOURCE
Original explanations, connected to the research.
Prototypical Networks for Few-shot Learning (Snell, Swersky & Zemel, 2017)Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (Finn, Abbeel & Levine, 2017)Language Models are Few-Shot Learners (Brown et al., 2020)Calibrate Before Use: Improving Few-Shot Performance of Language Models (Zhao et al., 2021)Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (Min et al., 2022)Human-level concept learning through probabilistic program induction (Lake, Salakhutdinov & Tenenbaum, 2015), the Omniglot paperWhat Can Transformers Learn In-Context? A Case Study of Simple Function Classes (Garg et al., 2022)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.