ClassroomThe classroom
Back to the lesson libraryMECHANISM · 13 MIN
M15.2 CONNECT THE MECHANISM

Attention: queries, keys, and values

How does the word "it" know whether to borrow meaning from "bird" or "road"? Build queries, keys, and values from scratch and score three tokens by hand.

LESSON OVERVIEW13 min lesson

Lesson overview

How does the word "it" know whether to borrow meaning from "bird" or "road"? Build queries, keys, and values from scratch and score three tokens by hand.

What you’ll explore

  • Distinguish query/key scoring from the weighted combination of value vectors.
GO TO THE SOURCE

Original explanations, connected to the research.

Attention Is All You Need, section 3.2 (Vaswani et al., 2017)Dive into Deep Learning — queries, keys, and valuesThe Illustrated Transformer (Jay Alammar, 2018)
Suggest a correction

A precise note can make an explanation better.

Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.

The file includes this note, the scene title, and lesson metadata. Your saved progress and quiz responses are excluded. Download before leaving or reloading to keep your note.