M10.3 CONNECT THE MECHANISM
Learn from a sample of examples at each step
Why read a million examples to take one step when thirty-two will do? See how random minibatches make training fast, noisy, and still accurate.
LESSON OVERVIEW11 min lesson
Lesson overview
Why read a million examples to take one step when thirty-two will do? See how random minibatches make training fast, noisy, and still accurate.
What you’ll explore
- Full-batch, stochastic, and minibatch descent use different gradient estimates; batch construction, averaging, and learning rate affect noise, cost, and convergence behavior.
GO TO THE SOURCE
Original explanations, connected to the research.
Dive into Deep Learning — authors’ open textbookA Stochastic Approximation Method (Robbins & Monro, 1951)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.