ClassroomThe classroom
Back to the lesson libraryMATH SIDE LESSON
B21 A SMALL STEP SIDEWAYS

Count work, memory, and parallel limits

Why does a prompt eight times longer make attention 64 times the work? Count steps and memory as inputs grow, and learn when big-O matters and when it doesn't.

LESSON OVERVIEW11 min lesson

Lesson overview

Why does a prompt eight times longer make attention 64 times the work? Count steps and memory as inputs grow, and learn when big-O matters and when it doesn't.

What you’ll explore

  • Count the steps and memory an algorithm needs as its input grows, compare growth rates with big-O, and explain why memory traffic and serial steps can limit speed.
GO TO THE SOURCE

Original explanations, connected to the research.

Dive into Deep Learning — authors’ open textbook
Suggest a correction

A precise note can make an explanation better.

Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.

The file includes this note, the scene title, and lesson metadata. Your saved progress and quiz responses are excluded. Download before leaving or reloading to keep your note.