Project: build and gate a tiny router
Train a small fixed-label router on 60 invented Tally messages, fit its temperature, then choose on validation the gate that decides which messages a machine may route alone. Open the test messages once, and report accuracy, ECE, automation, cost and a failure case.
Lesson overview
Train a small fixed-label router on 60 invented Tally messages, fit its temperature, then choose on validation the gate that decides which messages a machine may route alone. Open the test messages once, and report accuracy, ECE, automation, cost and a failure case.
What you’ll explore
- Build and evaluate a small softmax decision model on labelled messages. Measure accuracy and ECE, fit a temperature on a validation split, choose a confidence threshold for a target error rate on automated messages, and report automation rate, error counts, cost and one failure case from a test split used once.
Original explanations, connected to the research.
Introducing System One models and Jev (TypeSafe AI, late September 2026)On Calibration of Modern Neural Networks (Guo et al., 2017)Selective Classification for Deep Neural Networks (Geifman and El-Yaniv, 2017)Prepare a project review
Give someone the context to challenge your thinking.
Download the authored brief, deliverables, and review criteria with your notes. Use the worksheet for self or peer review. Project artifacts are ungraded here, and this review does not award mastery credit.
Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.