M13.4 CONNECT THE MECHANISM
Map one sequence into another
"The iced coffee" becomes "le café glacé": three words in, three out, in a different order. See how an encoder, a decoder, and attention pull that off, and why decoding needs a search.
LESSON OVERVIEW17 min lesson
Lesson overview
"The iced coffee" becomes "le café glacé": three words in, three out, in a different order. See how an encoder, a decoder, and attention pull that off, and why decoding needs a search.
What you’ll explore
- Sequence-to-sequence models encode an input and generate an output sequence; alignment, attention, and teacher forcing address different parts of that task.
GO TO THE SOURCE
Original explanations, connected to the research.
Sequence to Sequence Learning with Neural Networks (Sutskever, Vinyals & Le, 2014)Neural Machine Translation by Jointly Learning to Align and Translate (Bahdanau, Cho & Bengio, 2014)On the Properties of Neural Machine Translation: Encoder–Decoder Approaches (Cho et al., 2014)Sequence Level Training with Recurrent Neural Networks (Ranzato et al., 2015)Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks (Bengio et al., 2015)Dive into Deep Learning — sequence-to-sequence learningDive into Deep Learning — beam searchSuggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.