Signal-per-Token Optimization

Measure and maximize the ratio of useful context to total tokens sent to the model. The goal is not a larger context window but a higher density of relevant information per token.

How It Works

Instead of treating context size as the metric to optimize, teams measure how much of the prompt actually helps the assistant produce correct, reviewable, policy-aligned output. Every unnecessary token consumes compute, adds latency, and can dilute the model's attention. Every missing piece of context increases the odds of near-miss code that requires rework.

Techniques

Curated Context

Load only the files, sections, and metadata relevant to the current task. Avoid sending full directories or unrelated modules.

Structured Over Raw

Provide architecture diagrams, dependency maps, API contracts, and ownership metadata in structured form rather than expecting the model to infer them from raw code.

Governance Filters

Remove outdated documentation, deprecated APIs, and internal-only references that do not apply to the current change.

Task-Specific Subsetting

Tailor the context to the task type: a refactoring task needs different context than a bug investigation or test generation.

Measurement

• Track token volume per assistant request.
• Correlate token volume with output quality metrics: review comments, test failures, policy violations, rework rate.
• Benchmark assistants on the same task with varying context quality to isolate the signal-per-token effect.

Anti-patterns

• Equating a bigger context window with better context.
• Loading more raw code into the prompt to "be safe" without filtering for relevance.
• Measuring only adoption or speed without tracking output accuracy or rework cost.

See Also

• Selective Context Inclusion — primary technique for curating high-signal context.
• Token-Aware Formatting — reduces token waste from formatting overhead.
• Permission-Aware Context Retrieval — removes inaccessible noise that dilutes signal.
• Progressive Disclosure — the always-preloaded level must carry the highest signal per token of all.

Tools

• Tabnine Context Engine — improves signal per token by connecting assistants to structured, governed enterprise context instead of raw file dumps.