Compact before you overflow

Problem: Many agents compact only after the provider rejects a request as too long, then squash everything into a vague paragraph and lose exact paths, errors and decisions.
Idea: Before every model call, check whether the request still fits with room for the reply. If not, summarize only the older turns into a fixed template and keep recent turns verbatim.

Many agents handle long conversations reactively: send the request, get "context length exceeded" from the provider, then squash everything into a paragraph and retry. The summary is vague, the exact error string and file path from an hour ago are gone, and the agent starts guessing.

This repo's rewrite compacts before the request stops fitting, and it treats the summary as a structured hand-off.

Try it: when compaction kicks in

add turns, change the model window, make the summary fail

Before every model call, the estimated request is compared with the window minus room for the reply. Over it, older turns become a summary and the newest ~8k tokens stay word for word. If the summary fails, nothing changes.

When to compact

estimate({ system, messages, tools }) > context - max(maxOutputTokens, buffer)

Estimate the whole requestSystem prompt, messages and tool definitions, before calling the provider, not after it refuses.
Reserve room for the replyThe model's output budget or a 20,000-token buffer, whichever is larger. A request that fits but leaves no room to answer is still a failure.
Check the summary fits tooIf the summarization prompt itself wouldn't fit in the window minus its own output budget, it doesn't try.

What to keep, and how to summarize

Recent turns, verbatim

Walking back from the newest message, about 8,000 tokens are kept word for word, so the current task's details survive exactly.

Older turns, summarized

Everything before that goes to a summary with a fixed template: Objective, Important Details, Work State (Completed, Active, Blocked), Next Move, Relevant Files.

Preserve exact file paths, symbols, commands, error strings, URLs, and identifiers when known.

Tool results inside the summarization input are trimmed to 2,000 characters each, so one huge output can't crowd out the conversation being summarized.

Compacting again

The second compaction merges with the first, with explicit rules instead of "summarize this":

The <prior-summary> is discarded after this: anything you do not carry into the new summary is lost. [...] Where they conflict, the conversation wins.

Objectives, constraints and decisions are carried forward even if the recent conversation doesn't mention them; finished work moves from Active to Completed.

Failure changes nothing

The start of a compaction is recorded durably; progress is live-only. The switch to the summary happens only from a completed summary event. A failed or empty summarization returns without cutting over, the full history stays stored, and after a successful one the runner continues the original pending turn with a fresh context baseline.

Steal this

Check before every call whether the whole request still fits with room for the reply. Summarize only older turns, into a fixed template that preserves exact identifiers, keep recent turns verbatim, and only switch over once the summary exists.

Use it in your code

A prompt for your coding agent

Run it from the root of your repo. It checks whether the idea fits before it changes anything.

Add proactive context compaction to this app's LLM conversations.

The idea: before every model call, estimate the full request (system prompt, messages, tool definitions). If it no longer fits in the context window minus room for the reply, summarize the older turns into a fixed template, keep the most recent turns word for word, and switch to the summary only once it was produced successfully.

1. Find where conversation history is assembled for model calls, and what happens today when it gets too long (a provider error, silent truncation, or nothing).
2. If conversations here never get long, say so and stop.
3. Propose: a pre-call check (estimated request > context window - max(reply budget, safety buffer)); a split that keeps the newest turns up to a token budget verbatim; a summary prompt with fixed sections (objective, important details, work done, active, blocked, next move, relevant files) that must preserve exact paths, commands, errors and identifiers; and merge rules for updating an earlier summary (carry forward, the newer conversation wins on conflicts).
4. Implement it so a failed or empty summary changes nothing, and the original history stays stored. Test: a request just under and just over the limit, a second compaction that merges the first summary, and a failed summarization call.

Keep the estimate cheap. Approximate token counts are fine.

Read the source

Where this comes from

OpenCode (anomalyco/opencode), the open-source coding agent (these lessons are from its upcoming v2). The files behind this lesson: