M37.3 CONNECT THE MECHANISM
Learn a dictionary or a neighborhood-organized representation
Ask a model to describe photos using as few building blocks as possible, and edge detectors like those in your visual cortex appear on their own. Then make a map that sorts itself.
LESSON OVERVIEW14 min lesson
Lesson overview
Ask a model to describe photos using as few building blocks as possible, and edge detectors like those in your visual cortex appear on their own. Then make a map that sorts itself.
What you’ll explore
- Sparse coding reconstructs inputs with few active dictionary coefficients, while self-organizing maps update a winning prototype and its neighbors; their objectives and representation meanings differ.
GO TO THE SOURCE
Original explanations, connected to the research.
Emergence of simple-cell receptive field properties by learning a sparse code for natural images (Olshausen & Field, 1996)Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1? (Olshausen & Field, 1997)Self-Organized Formation of Topologically Correct Feature Maps (Kohonen, 1982)Towards Monosemanticity: Decomposing Language Models With Dictionary Learning (Bricken et al., 2023)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.