ClassroomThe classroom
Back to the lesson libraryMECHANISM · 12 MIN
M10.7 CONNECT THE MECHANISM

Keep signals and gradients in a usable range

Stack fifty layers and the signal fades to nothing or explodes into the tens of thousands. See how initialization, normalization, and number formats keep a deep network's numbers usable.

LESSON OVERVIEW12 min lesson

Lesson overview

Stack fifty layers and the signal fades to nothing or explodes into the tens of thousands. See how initialization, normalization, and number formats keep a deep network's numbers usable.

What you’ll explore

  • Initialization, normalization, activation choice, and numerical precision influence signal scale and optimization; stability requires inspecting their interaction rather than relying on one default.
GO TO THE SOURCE

Original explanations, connected to the research.

Dive into Deep Learning — authors’ open textbookDeep Learning — numerical computationDelving Deep into Rectifiers (He et al., 2015)Batch Normalization (Ioffe & Szegedy, 2015)Layer Normalization (Ba, Kiros & Hinton, 2016)
Suggest a correction

A precise note can make an explanation better.

Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.

The file includes this note, the scene title, and lesson metadata. Your saved progress and quiz responses are excluded. Download before leaving or reloading to keep your note.