Module 07 of 12

Tokens & Context Windows

Lessons

About This Module

Models don't read words, they read tokens: chunks of text that are often shorter or longer than a word. Everything the model sees in one request, including your instructions, any documents, the conversation so far and its own reply, has to fit inside the context window, the maximum number of tokens it can handle at once. Tokens also drive cost and speed, since APIs bill for input and output tokens separately.

The lessons cover what tokens are, how the context window fills up, and why a bigger window is not a guarantee of better answers. Long inputs can dilute the model's attention, and details buried in the middle are easier to miss. The practical habit to take away: choose what goes into the prompt on purpose, instead of pasting in everything you have.

Watch the lessons in order, then continue on to Context Engineering Basics.

Lessons

3 videos
01

Tokens & Context Window Explained for Beginners

A beginner's introduction to how models such as ChatGPT, Claude and Gemini split text into tokens, what the context window limit means, and why models can seem to forget earlier parts of a conversation.

Video 6min
02

LLM Tokens Explained: Why They Matter for Cost and Context

Why token counts don't map neatly to word counts, how some languages need more tokens than English, how context windows compare across models, and how input, output and reasoning tokens affect what you pay.

Video 8min
03

LLM Context Windows Explained: Why More Tokens Don't Always Mean Better Answers

Why a model can receive the information it needs and still miss it: attention dilution, lost-in-the-middle behavior, and irrelevant or conflicting context. Argues that carefully selected context usually beats maximal input.

Video 13min