M19.5 CONNECT THE MECHANISM
Let the current input control what the state retains
Ask a model to remember the colors and ignore every "um", and a fixed filter fails. Mamba lets each token set its own step size, which turns a state-space model into a learned gate.
LESSON OVERVIEW13 min lesson
Lesson overview
Ask a model to remember the colors and ignore every "um", and a fixed filter fails. Mamba lets each token set its own step size, which turns a state-space model into a learned gate.
What you’ll explore
- Selective state-space models make selected dynamics input-dependent, enabling content-sensitive state updates; hardware-aware scans and fixed-size recurrent state create a different tradeoff from attention.
GO TO THE SOURCE
Original explanations, connected to the research.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Gu & Dao, 2023)Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (Dao & Gu, 2024)An Empirical Study of Mamba-based Language Models (Waleffe et al., 2024)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.