M19.4 CONNECT THE MECHANISM
Represent a sequence through an evolving hidden state
A warm room remembers every burst of its heater, fading as it goes. That same idea, written as three small matrices, runs both as an RNN and as a convolution, and it grew into S4 and Mamba.
LESSON OVERVIEW12 min lesson
Lesson overview
A warm room remembers every burst of its heater, fading as it goes. That same idea, written as three small matrices, runs both as an RNN and as a convolution, and it grew into S4 and Mamba.
What you’ll explore
- State-space sequence models update a latent state and read outputs from it; continuous-to-discrete choices, stable dynamics, and structured computation determine their practical behavior.
GO TO THE SOURCE
Original explanations, connected to the research.
Efficiently Modeling Long Sequences with Structured State Spaces (Gu, Goel & Ré, 2021)HiPPO: Recurrent Memory with Optimal Polynomial Projections (Gu et al., 2020)Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Gu & Dao, 2023)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.