Hierarchical Memory

Separate agent memory into layers of varying resolution. The model queries the appropriate layer based on the current task, keeping the active context small while preserving access to deeper history.

Layers

• Working memory: high-resolution recent turns (raw conversation).
• Episodic memory: summaries of past conversation segments.
• Semantic memory: facts extracted and stored in a vector database for retrieval.

How It Works

When the agent needs context, it first checks working memory. If the information is not there, it queries episodic or semantic memory. This avoids inflating the prompt with irrelevant historical detail.

Trade-offs

• Pros: scalable to very long sessions; durable facts survive compaction.
• Cons: adds retrieval latency and implementation complexity; requires good indexing and query formulation.

See Also

• Rolling Summarization — episodic memory is often implemented as a rolling summary.
• Selective Context Inclusion — semantic memory retrieval is a form of selective inclusion.
• Compaction — hierarchical memory provides a structured alternative to compaction.

Tools

• MemGPT — hierarchical memory system with explicit memory management for LLM agents.
• LangChain Memory — implementations of buffer, summary, and vector-store-backed memory layers.