M01.7 CONNECT THE MECHANISM
Games, attention, and broadly reusable models
In 2016 a Go program played a move no expert would choose. Six years later ChatGPT reached a million users in five days. Trace how one pretrained model came to power thousands of products.
LESSON OVERVIEW12 min lesson
Lesson overview
In 2016 a Go program played a move no expert would choose. Six years later ChatGPT reached a million users in five days. Trace how one pretrained model came to power thousands of products.
What you’ll explore
- Distinguish a foundation model’s reusable training role from its architecture, adaptation method, and deployed application.
GO TO THE SOURCE
Original explanations, connected to the research.
Bommasani and colleagues — foundation models (2021)Vaswani and colleagues — Attention Is All You Need (2017)Silver et al. — Mastering the game of Go with deep neural networks and tree search (2016)Mikolov et al. — Efficient Estimation of Word Representations in Vector Space (2013)Devlin et al. — BERT (2018)Brown et al. — Language Models are Few-Shot Learners (GPT-3, 2020)Ouyang et al. — Training language models to follow instructions with human feedback (2022)Jumper et al. — Highly accurate protein structure prediction with AlphaFold (2021)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.