M37.7 CONNECT THE MECHANISM
Learn a vector field and let a solver define continuous depth
A ResNet is secretly Euler's method with a step size of 1. Shrink the step to zero and you get a network with no layers at all, just a learned rule for change and a solver.
LESSON OVERVIEW14 min lesson
Lesson overview
A ResNet is secretly Euler's method with a step size of 1. Shrink the step to zero and you get a network with no layers at all, just a learned rule for change and a solver.
What you’ll explore
- Neural ODEs parameterize a state derivative and integrate it with a numerical solver; solver tolerance, evaluation count, and gradient method affect accuracy, memory, and training behavior.
GO TO THE SOURCE
Original explanations, connected to the research.
Neural Ordinary Differential Equations (Chen, Rubanova, Bettencourt & Duvenaud, 2018)FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models (Grathwohl et al., 2019)Deep Residual Learning for Image Recognition (He, Zhang, Ren & Sun, 2015)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.