M37.2 CONNECT THE MECHANISM
Recover patterns through energy-based state dynamics
Smudge a stored letter and a network of simple units rebuilds it by rolling downhill in energy. The same idea, plus randomness, won a Nobel Prize and echoes inside transformer attention.
LESSON OVERVIEW15 min lesson
Lesson overview
Smudge a stored letter and a network of simple units rebuilds it by rolling downhill in energy. The same idea, plus randomness, won a Nobel Prize and echoes inside transformer attention.
What you’ll explore
- Hopfield-style associative memories update states toward stored attractors, while Boltzmann models use stochastic energy-based distributions; state inference, sampling, and parameter learning are distinct operations.
GO TO THE SOURCE
Original explanations, connected to the research.
Neural networks and physical systems with emergent collective computational abilities (Hopfield, 1982)A Learning Algorithm for Boltzmann Machines (Ackley, Hinton & Sejnowski, 1985)Storing infinite numbers of patterns in a spin-glass model of neural networks (Amit, Gutfreund & Sompolinsky, 1985)Training Products of Experts by Minimizing Contrastive Divergence (Hinton, 2002)Hopfield Networks is All You Need (Ramsauer et al., 2020)The Nobel Prize in Physics 2024, press releaseNeuronal DynamicsSuggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.