ClassroomThe classroom
Back to the lesson libraryMECHANISM · 15 MIN
M14.8 CONNECT THE MECHANISM

How a language model takes its next step

Every chatbot reply is built one token at a time from a list of probabilities. Work the numbers by hand, then turn the temperature, top-k, and top-p dials yourself.

LESSON OVERVIEW15 min lesson

Lesson overview

Every chatbot reply is built one token at a time from a list of probabilities. Work the numbers by hand, then turn the temperature, top-k, and top-p dials yourself.

What you’ll explore

  • An autoregressive language model predicts a distribution over the next token from context and generates by repeatedly selecting and appending tokens.
GO TO THE SOURCE

Original explanations, connected to the research.

The Curious Case of Neural Text Degeneration (Holtzman et al., 2019), nucleus (top-p) samplingSpeech and Language Processing, 3rd edition draft, chapters on language models and transformers (Jurafsky & Martin)Hierarchical Neural Story Generation (Fan, Lewis & Dauphin, 2018), top-k samplingAttention Is All You Need
Suggest a correction

A precise note can make an explanation better.

Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.

The file includes this note, the scene title, and lesson metadata. Your saved progress and quiz responses are excluded. Download before leaving or reloading to keep your note.