M09.6 CONNECT THE MECHANISM
Change one thing and measure what follows
Version 2 of a math tutor solves 78% of problems instead of 71%. Was it the new prompt, the bigger model, or luck? Design the experiment that tells you, and put error bars on it.
LESSON OVERVIEW13 min lesson
Lesson overview
Version 2 of a math tutor solves 78% of problems instead of 71%. Was it the new prompt, the bigger model, or luck? Design the experiment that tells you, and put error bars on it.
What you’ll explore
- Ablations, controls, paired comparisons, and uncertainty estimates help identify which changes explain a result, while confounding and selection limit causal interpretation.
GO TO THE SOURCE
Original explanations, connected to the research.
An Introduction to Statistical Learning — authors’ materialsAn Introduction to the Bootstrap (Efron & Tibshirani, 1993)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.