ClassroomThe classroom
Back to the lesson libraryMECHANISM · 14 MIN
M33.4 CONNECT THE MECHANISM

Notice when the production task changes

In December, holiday questions quadruple. Is the assistant failing, or just busy? Compute a drift score by hand, and learn what to check before anyone says "retrain."

LESSON OVERVIEW14 min lesson

Lesson overview

In December, holiday questions quadruple. Is the assistant failing, or just busy? Compute a drift score by hand, and learn what to check before anyone says "retrain."

What you’ll explore

  • Monitor a deployed model by separating input drift (measured with PSI), performance drift (measured with delayed labels), and pipeline faults, then choose between a fix, a scheduled update, and triggered retraining.
GO TO THE SOURCE

Original explanations, connected to the research.

Hidden Technical Debt in Machine Learning Systems (Sculley et al., NeurIPS 2015)The ML Test Score (Breck et al., 2017)Statistical Properties of Population Stability Index (Yurdakul, PhD dissertation, Western Michigan University, 2018)
Suggest a correction

A precise note can make an explanation better.

Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.

The file includes this note, the scene title, and lesson metadata. Your saved progress and quiz responses are excluded. Download before leaving or reloading to keep your note.