Chapter Three

Context
Engineering

AI doesn't forget — it was never
remembering in the first place.

Imagine you're having a conversation with a friend, except every few minutes someone erases the whiteboard you've both been writing on. You start over. Your friend looks at what's left and tries to keep going as if nothing happened. That's what it's like inside an AI's "brain" during a long conversation.

Here's the thing that surprises most people: AI has no memory. Not in the way you think of memory. It doesn't remember your conversation from yesterday. It doesn't even remember the beginning of this conversation if it's gone on long enough. Every single response it gives is based entirely on what's currently visible to it — a window of text called the context window.

Understanding this window — what goes into it, how big it is, and what happens when it fills up — is one of the most important skills in working with AI. This chapter is about becoming an engineer of that context.

The Whiteboard That Gets Erased

When you send a message to an AI, here is what actually happens: the system takes your entire conversation — every message you've sent, every response the AI gave — and feeds it all back to the model as a single block of text. The AI reads it all, generates the next response, and then... forgets everything.

Next time you send a message, the whole process repeats. The system pastes the entire conversation again, adds your new message, and the AI reads the whole thing fresh. It's like the world's most diligent but amnesiac reader — completely thorough each time, but starting from scratch every single turn.

Every response an AI generates is based entirely on what's in the context window at that moment. Nothing more. The context window is not memory. It's a whiteboard — and it has edges.

This has huge implications. It means the AI is only as good as the information currently in front of it. Leave out a critical detail? The AI won't magically recall it. Fill the window with noise? The AI will get confused. The context window is your most important resource. Learning to manage it is what separates casual users from people who get genuinely powerful results.

Tokens: The AI's Alphabet

AI doesn't read words the way you do. It reads tokens — chunks of text that are usually about three-quarters of a word. The word "incredible" is two tokens. "AI" is one token. A space before a word is often bundled into the same token.

Quick math

1 token ≈ 4 characters ≈ 0.75 words
100 tokens ≈ 75 words
A typical page of text ≈ 300 tokens
A full novel ≈ 100,000 tokens

Why does this matter? Because every AI model has a context window size measured in tokens. Claude's context window can hold 200,000 tokens — that's roughly a 500-page book. GPT-4o holds 128,000. These sound enormous, but they fill up faster than you'd think.

Every message you send, every response the AI writes, every system instruction working behind the scenes — they all consume tokens from the same shared budget. And when that budget runs out, things start falling off the edge.

The Context Window

Type messages and watch the window fill up

Context Window

System13 tok

You are a helpful, harmless, and honest assistant.

AI SEES THIS

Empty — type a message below

Visible: 13 tokens

Your Messages

Start typing messages below. Watch the context window fill up on the left. After 7 messages, the oldest ones will start to disappear -- that is the AI "forgetting."

Key insight: The system prompt stays pinned at the top. Everything else competes for space. Once the window fills up, early messages vanish from the AI's view.

When the Window Overflows

You're ten messages into a conversation. You mentioned your dietary restrictions in message three. By message fifteen, you're deep into restaurant recommendations. You ask: "Wait, what dietary restrictions did I mention?"

And the AI says: "I don't think you mentioned any specific dietary restrictions."

You feel frustrated. It's right there, I said it! But here's the thing: it's not right there. Not anymore. That early message may have slipped out of the visible window, or the model may have deprioritized it in favor of more recent context. The AI isn't being careless. It literally cannot see what's no longer in front of it.

This isn't a bug. It's architecture.

The AI processes the context window like a human reading a document in a single sitting. If the document is 100 pages long, details on page 3 might get fuzzy by page 95. The solution isn't a bigger window — it's smarter packing of what goes in.

The Forgetting Experiment

Same conversation, two approaches. Click through to see what happens.

MESSAGE 1 / 22

Standard Chat

I'm planning a trip and need help organizing everything.

With Context Summary

I'm planning a trip and need help organizing everything.

Click through the conversation. Watch what happens when we ask about dietary restrictions...

The System Prompt: Hidden Instructions

There's a part of every AI conversation you never see. Before your first message, before the AI says hello, there's a block of text sitting at the very top of the context window. It's called the system prompt, and it shapes everything that follows.

The system prompt is where the AI gets its personality, its rules, its boundaries. When ChatGPT refuses to help you build a weapon, that's the system prompt. When Claude says "I'd be happy to help," that's trained behavior reinforced by the system prompt. When a customer service chatbot stays on topic, that's — you guessed it — the system prompt.

Example system prompt

"You are a helpful cooking assistant. Only answer questions about cooking and food preparation. If asked about anything else, politely redirect the conversation back to cooking topics. Keep responses under 150 words. Use a warm, encouraging tone."

The system prompt is special because it stays pinned at the top of the context window. No matter how long the conversation gets, the system prompt doesn't scroll away. It's always there, always influencing the AI's behavior. That makes it the most valuable real estate in the entire context window.

Think of the system prompt as the AI's job description. It doesn't tell the AI what to say — it tells the AI who to be.

System Prompt Sandbox

Edit the system prompt, then test it with challenges

System Prompt58 tokens

Test Challenges

Chat Response

Click a challenge button on the left to send a test message. Then try editing the system prompt — remove the cooking restriction, or add new rules — and see how the responses change.

Try it: Delete the system prompt entirely and re-run the challenges. Notice how the AI responds to everything without boundaries. Then add your own rules — maybe "You are a fitness coach" or "Respond only in haiku." The system prompt is the AI's personality.

Strategies for Managing Context

Now that you understand the window, here's how to manage it like a pro. These are the techniques that separate people who "use AI" from people who get extraordinary results from it.

Summarize as you go

Every 5-10 messages, ask the AI to write a brief recap of key decisions and details. Then paste that summary into your next message. This "resets" the context with the most important information front and center.

Front-load what matters

Put the most critical information at the beginning of your message, not buried at the end. AI pays more attention to what comes first and last — the middle can get lost, especially in long prompts.

Start fresh strategically

Don't be afraid to start a new conversation when the topic shifts significantly. Carry over a summary of what you decided, not the full history. A clean context is often better than a cluttered one.

Be explicit about what to remember

"Important: the user is vegetarian. Do not forget this." Sounds silly, but it works. Explicit reminders act as anchors that the AI weighs more heavily.

Use structured formats

Bulleted lists, headers, and clear labels help the AI parse your context more efficiently. A well-organized prompt is worth twice its token count compared to a wall of prose.

Context engineering is the new prompt engineering.

Writing a good prompt is the first step. But deciding what information to put in front of the AI — and how to structure it — is where the real leverage is. The best prompt in the world fails if the context is wrong.

Key Concepts

Context ≠ Memory

AI has no persistent memory by default. What feels like ‘forgetting’ is just the context window filling up.