Prompt Truncation

Cut the input at a fixed token or character limit. Fast but can drop critical context. Best used when the tail of the input is least important, such as the oldest conversation turns.

How It Works

The prompt is measured in tokens or characters. Everything beyond the limit is discarded. The truncation point can be at the end (drop oldest), at the beginning (drop newest), or at a custom boundary.

Trade-offs

• Pros: extremely fast; deterministic; no extra model calls.
• Cons: can silently drop critical information; no awareness of semantic importance.

When to Use

As a last-resort safety valve when the prompt exceeds the context window, or as a coarse pre-filter before more expensive compression techniques.

See Also

• Prompt Summarization — higher-quality alternative to truncation.
• Sliding Window — truncation applied specifically to conversation history.
• Compaction — truncation is the coarsest form of compaction.
• Output Spooling to Disk — spooling keeps the dropped tail retrievable on disk instead of losing it.

Tools

• tiktoken — fast tokenizer for measuring prompt length before truncation.
• LangChain ConversationBufferWindowMemory — implicit truncation via window size.