YOUR FOUNDATION-FIRST SYLLABUS
Multimodal systems
Bring together words, pictures, and sound in one system. Learn how their stored forms connect, how to combine their clues, and what to test when one source is missing or contradicts another.
0 of 95 core ideas demonstrated94 lessons1220 estimated minutes remaining
Already know some of this?
Go straight to an idea’s knowledge check. Passing its different checks without hints carries that evidence into this path. Checking an advanced idea does not award its prerequisites; you can still explore them.
Check what I already knowTHE IDEAS, IN ORDER
Shared credit is automaticBuild your understanding.
Chapter counts use compatible knowledge-check evidence. Reading a lesson does not mark its ideas as demonstrated.
Chapter 1 · Lessons 1–7 Meet an AI system Start with an everyday task, then follow its inputs, model, and output. 7 lessons · 0 of 7 core ideas demonstrated Up next
01
What is AI? Start with an everyday task Common Ground · 11 min
To explore02 Inputs and outputs: what goes in and what comes out Common Ground · 13 min
To explore03 Data and examples: what a computer can learn from Common Ground · 10 min
To explore04 What is a model? A small rule inside a bigger app Common Ground · 10 min
To explore05 Weights: the adjustable numbers in a model Common Ground · 12 min
To explore06 Training and inference: changing a rule or using it Common Ground · 13 min
To explore07 Follow one small AI system from start to finish Common Ground · 9 min
To exploreChapter 2 · Lessons 8–14 How computers represent a problem Explore everyday AI ideas; how AI got here; how computers represent a problem; and supporting ideas. 7 lessons · 0 of 7 core ideas demonstrated
08
Different ways to solve the same problem Common Ground · 11 min
To explore09 How to tell whether an AI answer is useful Common Ground · 10 min
To explore10 The questions that shaped AI Common Ground · 15 min
To explore11 How a computer stores a picture or a sentence Learning mechanisms · 16 min
To explore12 Describe a computation so someone else can follow it Learning mechanisms · 11 min
To explore13 What a representation makes easy to learn Learning mechanisms · 12 min
To explore14 From the world to a dataset Learning mechanisms · 11 min
To exploreChapter 3 · Lessons 15–21 Learning to predict Explore learning to predict; helping a model improve; math preparation when it is needed. 7 lessons · 0 of 8 core ideas demonstrated
15
The numbers you actually need Math preparation · 10 min
To explore16 A function is a rule Math preparation · 9 min
To explore17 Build a tiny prediction model Learning mechanisms · 11 min
To explore18 How does a model improve a prediction? Learning mechanisms · 32 min
To explore19 Read an equation one symbol at a time Math preparation · 10 min
To explore20 A derivative is a local change Math preparation · 13 min
To explore21 The chain rule, one step at a time Math preparation · 13 min
To exploreChapter 4 · Lessons 22–28 Inside a neural network: How an error reaches earlier weights Explore inside a neural network; math preparation when it is needed. 7 lessons · 0 of 7 core ideas demonstrated
22
How an error reaches earlier weights Learning mechanisms · 11 min
To explore23 Inside an artificial neuron Learning mechanisms · 14 min
To explore24 Why layers need more than multiplication Learning mechanisms · 12 min
To explore25 Vectors: lists that work together Math preparation · 10 min
To explore26 Follow shapes through a numerical layer Math preparation · 10 min
To explore27 Trace inputs through layers to a loss Learning mechanisms · 17 min
To explore28 Let the graph carry derivatives Learning mechanisms · 12 min
To exploreChapter 5 · Lessons 29–35 Helping a model improve Explore helping a model improve; math preparation when it is needed. 7 lessons · 0 of 7 core ideas demonstrated
29
Probability without the mystery Math preparation · 10 min
To explore30 Averages over uncertain outcomes Math preparation · 9 min
To explore31 Learn from a sample of examples at each step Learning mechanisms · 11 min
To explore32 Several inputs, several rates of change Math preparation · 9 min
To explore33 Choose a direction and a step size Math preparation · 9 min
To explore34 How momentum and Adam use earlier gradients Learning mechanisms · 15 min
To explore35 Constrain a solution or change the update Learning mechanisms · 12 min
To exploreChapter 6 · Lessons 36–42 Inside a neural network: Help information and learning signals travel Explore helping a model improve; inside a neural network; learning from useful data; and supporting ideas. 7 lessons · 0 of 7 core ideas demonstrated
36
Powers and logarithms, one step at a time Math preparation · 11 min
To explore37 When the computer cannot store the exact number Math preparation · 12 min
To explore38 Keep signals and gradients in a usable range Learning mechanisms · 12 min
To explore39 Help information and learning signals travel Learning mechanisms · 12 min
To explore40 Connect data, gradients, and saved model state Learning mechanisms · 15 min
To explore41 Different ways to learn Common Ground · 12 min
To explore42 Create a learning task from the data itself Learning mechanisms · 14 min
To exploreChapter 7 · Lessons 43–50 Connecting words, images, and other signals: Connect information expressed in different forms Explore inside a neural network; making sense of images; learning from sequences and sound; and supporting ideas. 8 lessons · 0 of 8 core ideas demonstrated
43
What can a network’s hidden features tell us? Learning mechanisms · 12 min
To explore44 Distances, neighborhoods, and transformations Math preparation · 12 min
To explore45 What changes when an image changes? Learning mechanisms · 13 min
To explore46 From a waveform to time-frequency features Learning mechanisms · 14 min
To explore47 Connect information expressed in different forms Learning mechanisms · 15 min
To explore48 Bring matching images and descriptions closer in representation space Learning mechanisms · 17 min
To explore49 How text becomes tokens Learning mechanisms · 13 min
To explore50 From token IDs to learned vectors Learning mechanisms · 14 min
To exploreChapter 8 · Lessons 51–57 Following attention through a transformer Explore following attention through a transformer; learning from sequences and sound; turning language into model inputs. 7 lessons · 0 of 7 core ideas demonstrated
51
Attention: queries, keys, and values Learning mechanisms · 13 min
To explore52 From attention scores to a causal mask Learning mechanisms · 17 min
To explore53 Run several learned comparisons in parallel Learning mechanisms · 10 min
To explore54 Carry a state from one step to the next Learning mechanisms · 14 min
To explore55 How a language model takes its next step Your system, connected · 15 min
To explore56 Map one sequence into another Learning mechanisms · 17 min
To explore57 Match information flow to the learning objective Learning mechanisms · 13 min
To exploreChapter 9 · Lessons 58–65 Connecting words, images, and other signals: Connect a vision encoder to a language model Explore connecting words, images, and other signals; how computers represent a problem; following attention through a transformer; and supporting ideas. 8 lessons · 0 of 8 core ideas demonstrated
58
Connect a vision encoder to a language model Learning mechanisms · 14 min
To explore59 Trace an example through a batch Learning mechanisms · 12 min
To explore60 Turn images, sound, and video into manageable model inputs Learning mechanisms · 14 min
To explore61 Follow one token through a transformer Your system, connected · 13 min
To explore62 Inside a language model training step Learning mechanisms · 14 min
To explore63 What can a small experiment tell us? Math preparation · 10 min
To explore64 Clean data without erasing the problem Learning mechanisms · 12 min
To explore65 Protect the examples used to judge a model Learning mechanisms · 15 min
To exploreChapter 10 · Lessons 66–72 Preparing and training a language model Explore preparing and training a language model; turning language into model inputs; learning from useful data. 7 lessons · 0 of 7 core ideas demonstrated
66
Assemble token batches without leaking targets Learning mechanisms · 14 min
To explore67 Choose what a pretraining prediction means Learning mechanisms · 13 min
To explore68 Words, structure, meaning, and context Learning mechanisms · 12 min
To explore69 Represent text with counts and short contexts Learning mechanisms · 14 min
To explore70 Learn the units used by a language model Learning mechanisms · 15 min
To explore71 Choose the model that pretraining will fit Learning mechanisms · 15 min
To explore72 Where did this dataset come from? Learning mechanisms · 12 min
To exploreChapter 11 · Lessons 73–80 Testing what a model has learned Explore preparing and training a language model; learning to predict; testing what a model has learned. 8 lessons · 0 of 8 core ideas demonstrated
73
Build a traceable corpus before spending training compute Learning mechanisms · 19 min
To explore74 Choose how much the model sees of each source Learning mechanisms · 13 min
To explore75 Turn a linear score into a class probability Learning mechanisms · 11 min
To explore76 Control complexity without peeking at the answer Learning mechanisms · 12 min
To explore77 Will the model work on new examples? Learning mechanisms · 14 min
To explore78 Why a simpler model can predict better Learning mechanisms · 14 min
To explore79 Evaluate the procedure that chose the model Learning mechanisms · 14 min
To explore80 More capacity changes more than one thing Learning mechanisms · 14 min
To exploreChapter 12 · Lessons 81–87 Adapting a model after pretraining: Track what each adaptation stage changes Explore preparing and training a language model; following attention through a transformer; adapting a model after pretraining; and supporting ideas. 7 lessons · 0 of 7 core ideas demonstrated
81
Allocate compute and preserve useful checkpoints Learning mechanisms · 15 min
To explore82 Give a transformer information about order Learning mechanisms · 12 min
To explore83 Continue pretraining without losing sight of the starting model Learning mechanisms · 12 min
To explore84 Track what each adaptation stage changes Learning mechanisms · 15 min
To explore85 Learn response behavior from demonstrations Learning mechanisms · 14 min
To explore86 Learn a scoring signal from response comparisons Learning mechanisms · 14 min
To explore87 Measure surprise and compare distributions Math preparation · 12 min
To exploreChapter 13 · Lessons 88–94 Adapting a model after pretraining: Learn from feedback while controlling policy changes Explore adapting a model after pretraining; testing what a model has learned; connecting words, images, and other signals. 7 lessons · 0 of 7 core ideas demonstrated
88
Learn from feedback while controlling policy changes Learning mechanisms · 14 min
To explore89 Train on preference pairs without a separate online reward loop Learning mechanisms · 13 min
To explore90 Transfer behavior through teacher outputs and selected examples Learning mechanisms · 13 min
To explore91 Change one thing and measure what follows Learning mechanisms · 13 min
To explore92 Measure adaptation gains without overlooking regressions Learning mechanisms · 14 min
To explore93 Train alignment, generation, and instruction behavior across modalities Learning mechanisms · 14 min
To explore94 Check whether answers actually follow the supplied media Learning mechanisms · 14 min
To exploreOptional extensions and comparisons (6)
These lessons deepen or compare the reference design. They do not add requirements to this path’s completion.
How to use this school without getting lostOptional orientation: how paths, side lessons, and practice fit together, plus study habits that make learning stick.Practice, progress, and knowing what you understandOptional orientation: use practice, mistakes, and spaced review to find out what you really understand.Misuse in practice: jailbreaks, deepfakes, and provenanceA familiar face or voice is no longer proof of who is speaking. Learn how misuse works and the checks that still hold.Model motion and persistence across framesAdd time to vision: how models understand and generate video.Predict what may happen next, with or without actionsModels that predict what happens next, for planning and generation.Connect visual instructions to a robot’s action interfaceRobots that follow spoken instructions: connect vision and language to motor actions.