M08.6 CONNECT THE MECHANISM
Create a learning task from the data itself
Nobody labeled the text word2vec learned from, yet it discovered that king − man + woman lands near queen. See how models invent their own training puzzles, and how those puzzles can backfire.
LESSON OVERVIEW14 min lesson
Lesson overview
Nobody labeled the text word2vec learned from, yet it discovered that king − man + woman lands near queen. See how models invent their own training puzzles, and how those puzzles can backfire.
What you’ll explore
- Self-supervised learning makes its own targets from unlabeled data; test the learned features on separate tasks to see if they help.
GO TO THE SOURCE
Original explanations, connected to the research.
Dive into Deep Learning — authors’ open textbookA Simple Framework for Contrastive Learning of Visual Representations (SimCLR; Chen et al., 2020)Efficient Estimation of Word Representations in Vector Space (word2vec; Mikolov et al., 2013)Representation Learning with Contrastive Predictive Coding (van den Oord et al., 2018)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.