M09.4 CONNECT THE MECHANISM
Test the rule where conditions change
A wildlife camera model scores 94% at the sites it trained on and 71% in the next valley. Learn to name the kind of change, test for shortcuts, and fix the arithmetic.
LESSON OVERVIEW14 min lesson
Lesson overview
A wildlife camera model scores 94% at the sites it trained on and 71% in the next valley. Learn to name the kind of change, test for shortcuts, and fix the arithmetic.
What you’ll explore
- Test how predictions change under specific shifts in the data; passing one shift doesn't prove a model handles others.
GO TO THE SOURCE
Original explanations, connected to the research.
An Introduction to Statistical Learning — authors’ materialsDive into Deep Learning — environment and distribution shiftRecognition in Terra Incognita (Beery, Van Horn & Perona, 2018)Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs (Zech et al., 2018)Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (Hendrycks & Dietterich, 2019)WILDS: A Benchmark of in-the-Wild Distribution Shifts (Koh et al., 2021)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.