ClassroomThe classroom
Back to the lesson libraryMECHANISM · 12 MIN
M10.6 CONNECT THE MECHANISM

Read the shape of a loss landscape

Your loss sits flat for an hour, then suddenly plunges. Learn to picture the terrain training walks across (bowls, ravines, saddles, and plateaus) and what each does to your steps.

LESSON OVERVIEW12 min lesson

Lesson overview

Your loss sits flat for an hour, then suddenly plunges. Learn to picture the terrain training walks across (bowls, ravines, saddles, and plateaus) and what each does to your steps.

What you’ll explore

  • Convexity, curvature, stationary points, and saddles describe objective geometry; a two-dimensional plot is only a slice through a high-dimensional training problem.
GO TO THE SOURCE

Original explanations, connected to the research.

Deep Learning — numerical computationDive into Deep Learning — authors’ open textbookIdentifying and attacking the saddle point problem in high-dimensional non-convex optimization (Dauphin et al., 2014)Visualizing the Loss Landscape of Neural Nets (Li et al., 2018)
Suggest a correction

A precise note can make an explanation better.

Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.

The file includes this note, the scene title, and lesson metadata. Your saved progress and quiz responses are excluded. Download before leaving or reloading to keep your note.