Context Window Management
Long conversations quietly destroy transactional data. Learn the structural patterns — persistent fact blocks, tool trimming, and caching — that keep Claude coherent.
Context window management is the foundation of every reliable Claude system. The failure mode is rarely a hard token overflow — it is quiet degradation, where important details survive syntactically but lose their weight as the window fills. Because the Claude API is stateless, each request must carry the full conversation, so how you structure and prune that payload determines whether the model still knows the order number ten turns later.
The progressive summarisation trap
The intuitive way to fit a long conversation is to summarise older turns. This is exactly where transactional data dies. A precise customer statement like I'd like a refund of $247.83 for order #8891 placed on March 3rd collapses into customer wants a refund for a recent order — the amount, order ID, and date are gone. The fix is not a better summary; it is to stop summarising the facts at all.
Extract transactional facts — amounts, dates, order numbers, statuses — into a structured case-facts block that is injected into every prompt, outside the summarised history. The narrative can be compressed; the facts are pinned verbatim.
{
"case_facts": {
"customer_id": "C-40921",
"order_id": "8891",
"order_date": "2024-03-03",
"refund_amount_usd": 247.83,
"item": "Wireless keyboard (returned, unopened)",
"status": "refund_requested"
}
}Let older turns be progressively summarised along with the transactional data. Amounts, order IDs, and dates collapse into vague prose and are gone.
# summary of earlier turns
"Customer wants a refund for a recent order."Pin the facts verbatim in a case-facts block injected outside the summary. Compress the narrative, never the numbers.
# case_facts (never summarised)
{ "order_id": "8891", "refund_amount_usd": 247.83 }Pinning facts in a persistent block that bypasses summarisation is a structural guarantee. Telling the model 'remember the amount' is a request it can ignore under context pressure. On the exam, prefer the structural fix.
Lost in the middle
Models weight the beginning and end of a long input more reliably than the middle. When you aggregate many tool results or documents, burying the conclusion in the center means it gets underweighted. Place a key-findings summary at the top of aggregated inputs, use explicit section headers throughout, and put detailed evidence after. This is a layout fix, not a prompt fix — no amount of 'pay attention to section 4' compensates for bad positioning.
Trim tool results before they accumulate
A single order-lookup tool might return 40+ fields — timestamps, warehouse codes, carrier IDs — when the agent needs five. Left untrimmed, that verbosity is re-sent on every turn, compounding token cost and diluting attention. Trim verbose outputs down to the relevant fields at the point of ingestion.
def trim_order(raw: dict) -> dict:
# Keep only what the agent reasons over; drop the other 35+ fields.
return {
"order_id": raw["order_id"],
"date": raw["created_at"],
"amount": raw["total_usd"],
"return_eligible": raw["returns"]["eligible"],
"item": raw["line_items"][0]["title"],
}Prompt caching
Caching is the complement to trimming: trimming removes tokens, caching makes reused tokens cheap. Put static content first — system instructions, tool definitions, reference documents — set a cache_control breakpoint at the end of it, then place dynamic content (the current user turn) after. The API reuses the cached prefix at a fraction of the cost.
If any dynamic content sits before your static block, the prefix changes every request and caching fails entirely. The order is non-negotiable: static first, breakpoint, then dynamic.
Put the dynamic user turn or a timestamp before the static system prompt. The prefix changes every request, so nothing is ever reused.
messages = [ user_turn, system_prompt, tools ] # prefix churnsPut static system instructions and tool definitions first, set the cache_control breakpoint at their end, then append the dynamic turn.
messages = [ system_prompt, tools, <cache_control>, user_turn ]| Problem | Wrong fix | Right fix |
|---|---|---|
| Facts lost in long chats | Better summaries | Persistent case-facts block |
| Middle content ignored | Instruct model to focus | Key findings first + headers |
| Token bloat | Bigger context window | Trim tool results at ingest |
| High cost on static content | Fewer instructions | cache_control on static prefix |
There is no server-side session. If a message is not in your request array, Claude cannot see it. Never selectively truncate history hoping the model 'remembers' — use fact blocks plus summarisation instead.
How the exam will try to trick you
The distractors below look right under time pressure — learn the tell.
- The trap
Progressive summarisation is safe for transactional data as long as the summary is well written.
Correct answerPin amounts, dates, and IDs verbatim in a case-facts block that lives outside the summarised history.
Why: Summarisation systematically destroys numbers, dates, and identifiers — no summary is careful enough.
- The trap
Fix lost-in-the-middle by telling the model to
pay attention to section 4.Correct answerPlace the key findings at the start of aggregated inputs and add explicit section headers.
Why: Prompt reminders are unreliable for position effects; the fix is layout, not instruction.
- The trap
Keep the full 40+ field tool output in history in case the model needs it later.
Correct answerTrim tool results to the few relevant fields at ingestion, before they accumulate.
Why: Untrimmed outputs are re-sent every turn, exhausting the token budget and diluting attention.
- The trap
Selectively drop older messages to save tokens and trust the model to remember.
Correct answerKeep the full history you want seen; compress narrative via summary plus a fact block.
Why: The API is stateless — a message absent from the request array simply does not exist for Claude.
Key takeaways
- Summarisation destroys transactional data — pin amounts, dates, and IDs in a case-facts block outside the summary.
- The Claude API is stateless; every request must include the full conversation history you want the model to see.
- Combat lost-in-the-middle by placing key findings at the start of aggregated inputs, not by instructing the model.
- Trim verbose tool outputs to only relevant fields before they accumulate across turns.
- Prompt caching requires static content first with a cache_control breakpoint before dynamic content.
- Prefer structural fixes over larger context windows — a bigger window still fills with the same noise.
Frequently asked questions
How do you manage Claude's context window in long conversations?+
Separate durable facts from disposable narrative. Extract transactional data (amounts, dates, order IDs, statuses) into a structured case-facts block that is prepended to every request, summarise the conversational history around it, and trim verbose tool outputs to the few fields you actually reason over. Because the API is stateless, this block is your only guarantee that critical values survive; a larger context window alone does not prevent degradation.
Why does prompt caching fail even when I set cache_control?+
Caching keys on a stable prefix. If dynamic content — the current user message or a timestamp — appears before your static system instructions and tool definitions, the prefix changes on every request and no cache hit is possible. Fix the layout: static content first, cache_control breakpoint at its end, then dynamic content.