Compaction

Compaction is how a coding agent keeps working after its conversation history becomes too large to send on every turn. The core idea is simple: replace a long raw transcript with a smaller representation that preserves the state needed to continue the task.

Without compaction, the prompt keeps growing until it becomes too expensive, too slow, or too large for the model context window.

Techniques

Sliding Window

Keep the newest messages and drop the oldest ones. Cheap, deterministic, and easy to implement, but loses early decisions and is fragile on long debugging or refactoring tasks. Usually a fallback, not the best standalone strategy. See Sliding Window.

Summarization

Summarize older context into a shorter natural-language state and continue from that summary plus recent messages. Typical summary contents: current goal, files touched, decisions made, unresolved issues, next steps. This is the most common practical approach in production agents. See Prompt Summarization and Rolling Summarization.

Structured State Extraction

Instead of a loose summary, extract a schema-like state object. Example fields: task, assumptions, edited files, failing tests, open questions, pending actions. This reduces the chance that important state is omitted during compaction. See Structured State Extraction.

Extractive Prompt Compression

Remove low-value tokens or spans from the prompt while keeping high-value content such as code, errors, or instructions. Usually a supplement to agent memory design rather than the whole solution. See Prompt Truncation.

External Memory Extraction

Move durable facts out of the live prompt and store them for later retrieval. Examples: repository conventions, user preferences, architectural decisions, stable bug context. The prompt gets smaller because not everything has to remain inline. See Hierarchical Memory.

Learned Self-compaction

Train the model or agent policy to produce better compacted state automatically. The strongest long-horizon approach in principle, but requires training infrastructure and is less common than prompt-based summarization.

What Good Compaction Keeps

A good compacted state should preserve:

• the actual task objective
• important constraints from the user
• the current plan
• what changed in the code
• what was tried and failed
• what remains unresolved
• the immediate next action

If the compacted state drops any of these, the agent may repeat work or make inconsistent decisions.

When to Compact

Compaction is best done at a stable boundary:

• after a subtask is completed
• after a test run is interpreted
• after a file edit batch
• after a bug hypothesis is resolved

Compacting in the middle of an unresolved reasoning chain is riskier because the summary may flatten important uncertainty.

Compaction and Cache Behavior

Compaction usually breaks input-prefix caching on the inference server. Input caching depends on the request prefix matching earlier requests; compaction replaces the old transcript with a new summary or state block, changing the token prefix so prior cached prefill often cannot be reused.

Cache reuse starts over from the new compacted state. Two exceptions:

• if the server supports block-level prefix reuse, unchanged leading segments may still be cached
• if the agent keeps a stable system prompt or repository context ahead of the compacted state, that stable prefix may still benefit from caching

In practice, compaction trades some cache hit rate for a much smaller prompt. For long-running coding sessions, that trade is usually worth it once the raw transcript becomes large enough.

Practical Design

For most coding agents, the strongest pragmatic design is:

1. keep a small recent working window
2. summarize or structure older history
3. store durable facts in external memory
4. compact only at coherent task boundaries

That combination is simpler than learned self-compaction and much more robust than naive truncation.

See Also

• Sliding Window — simplest compaction technique.
• Rolling Summarization — most common compaction technique.
• Prompt Caching — compaction usually breaks the prompt cache.
• Cache-aware Routing — compaction creates a natural cache boundary for model switching.
• Incremental Plan Updates — the plan is part of what must be preserved during compaction.
• Structured State Extraction — schema-based compaction that avoids the O(T²) token growth of transcript-based memory.
• Context Isolation via Subagents — both keep raw bytes out of the working transcript: compaction after they arrive, isolation before.
• Output Spooling to Disk — keeping log dumps out of the transcript delays the need to compact at all.

Tools

• MemGPT — hierarchical memory system with explicit memory management for LLM agents, including compaction and external memory.
• LangChain ConversationSummaryMemory — summarizes conversation history to keep context bounded.
• Aider — coding assistant that manages repository context and conversation history with summarization and structured state.