M28.8 CONNECT THE MECHANISM
Make conclusions match the strength of the experiment
The new model scored 7 points higher, on an easier month of chats. Learn to change one thing at a time, report a "no effect" honestly, and write the plan before the scores come in.
LESSON OVERVIEW11 min lesson
Lesson overview
The new model scored 7 points higher, on an easier month of chats. Learn to change one thing at a time, report a "no effect" honestly, and write the plan before the scores come in.
What you’ll explore
- Causal evaluation changes controlled factors and tests alternatives; reproducible suites and negative results constrain hypotheses without turning limited evidence into universal claims.
GO TO THE SOURCE
Original explanations, connected to the research.
On the State of the Art of Evaluation in Neural Language Models (Melis et al., 2017)Are GANs Created Equal? A Large-Scale Study (Lucic et al., 2017)Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses (Lakens, 2017)Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations (Miller, 2024)Deep Reinforcement Learning that MattersHELMSuggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.