ClassroomThe classroom
Back to the lesson libraryMECHANISM · 12 MIN
M11.5 CONNECT THE MECHANISM

Let the graph carry derivatives

One line of PyTorch, loss.backward(), finds the gradient of every weight in a giant model. Build the 30-line engine behind that line and learn its two classic traps.

LESSON OVERVIEW12 min lesson

Lesson overview

One line of PyTorch, loss.backward(), finds the gradient of every weight in a giant model. Build the 30-line engine behind that line and learn its two classic traps.

What you’ll explore

  • Distinguish automatic differentiation from finite differences and symbolic algebra, and diagnose accumulation or graph-detachment errors.
GO TO THE SOURCE

Original explanations, connected to the research.

PyTorch — automatic differentiation fundamentalsDive into Deep Learning — forward/backward computationBaydin et al. — Automatic Differentiation in Machine Learning: a Survey (JMLR, 2018)Karpathy — micrograd, a tiny scalar autograd engine
Suggest a correction

A precise note can make an explanation better.

Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.

The file includes this note, the scene title, and lesson metadata. Your saved progress and quiz responses are excluded. Download before leaving or reloading to keep your note.