Meta-prompting

Use a model to rewrite a prompt into a shorter one, and accept a candidate only if a behavioral test suite still passes. The prompt is treated like code under refactoring: the model proposes smaller versions, the tests are the specification, and the token count is the objective being minimized.

The targets that pay most are the prompts resident on every turn — system instructions, tool descriptions, tool schemas, agent definitions. They accrete over months as each bug gets its own sentence, and nobody dares delete a sentence because nobody knows which behavior it holds up.

How It Works

1. Collect the resident prompt surface for one component: its tool description, schema docs, agent definitions, the system-prompt paragraphs that mention it, and companion tools that restate the same rules.
2. Write behavioral tests for every requirement the prompt must keep: given this situation, the agent must (or must not) do X. These are evals over agent trajectories, not string checks on the prompt.
3. Ask a model for a smaller candidate, giving it the current prompt, the requirements, and the failures of the previous candidate.
4. Run the tests on the candidate. Keep it if it passes and is shorter; feed failures back otherwise.
5. Iterate until the size stops shrinking or tests start failing, then ship behind an online experiment.

The loop is "meta" because the model's output is a prompt, and the quality of that prompt is judged by how another model run behaves under it.

Why It Works

• Accreted prompts are redundant. The same rule is often stated in the tool description, the system prompt, and an agent definition; a rewrite that sees all three can state it once.
• Enumerations compress into principles. Allowlists and denylists written case by case can often be replaced by the rule that generated them. GitHub Copilot replaced explicit lists of which agents may run concurrently with one sentence: "Independent agents can run in parallel; consider side effects."
• Per-turn prompts multiply. A token removed from a resident prompt is removed from every turn of every session. Even with Prompt Caching, cached tokens are still billed at a discount and still occupy the context window.

Evidence

GitHub Copilot applied this to the guidance of its task tool (the subagent launcher), which had spread across tool descriptions, schemas, agent definitions, system instructions, and companion tools. The loop halved the prompt, removing about 1,300 tokens per turn: 1.8% fewer prompt tokens per session and 2.9% lower normalized cost per active hour, with no quality regression detected (GitHub blog, 2026).

The first attempt shipped a regression the offline tests missed: independent custom agents started running sequentially instead of in parallel. Users found it; the team wrote a regression eval for that behavior before compressing further. Their lesson: "Prompt behavior needs tests. If a behavior is not tested, a shorter prompt can remove it."

The Test Suite Is the Real Artifact

The compressor optimizes against the tests, so anything untested is, from its point of view, free to delete. Consequences:

• The token savings are bounded by test coverage, not by the model's cleverness. A weak suite yields an aggressive prompt that silently drops behaviors.
• Each production regression becomes a new test, which makes the next compression pass safer. The suite outlives any particular prompt.
• Tests must exercise behavior (trajectories, tool calls, ordering), not the presence of phrases in the prompt — the whole point is that the phrasing changes.

Limitations

• Evals over agent trajectories are expensive; the compression loop spends tokens up front to save them per turn, so it pays only for high-traffic prompts.
• Behavioral tests are nondeterministic; a candidate needs several runs per test to distinguish a real regression from noise.
• Changing a resident prompt invalidates its cache once for every user (see Prefix Stability); ship compressed prompts together with other prompt changes rather than one by one.
• Shorter is not always cheaper: a prompt compressed past clarity can make the agent take extra turns to recover, costing more per task than it saves per turn.

Terminology

"Meta-prompting" is also used in the literature for a different idea — a conductor model that decomposes a task and prompts expert instances of itself (Suzgun & Kalai, 2024). This article uses the term in the prompt-optimization sense: a model writing and refining prompts, as in APE and OPRO below, with token count as the objective.

See Also

• Model-Migration Prompt Pruning — the manual, migration-triggered cousin: remove instructions a newer model no longer needs. Meta-prompting automates the pruning and can run at any time.
• Token-Aware Formatting — removes presentation waste deterministically; meta-prompting removes semantic redundancy, which needs a model and tests.
• Prompt Summarization — compresses input content per request; meta-prompting compresses instructions once, offline.
• Dynamic Tool Loading — removes tool definitions that are not needed; meta-prompting shrinks the ones that are.
• Effort Calibration — same discipline: decide from measured cost/quality, not from intuition.

Tools

• How we make AI coding more cost-efficient without sacrificing task quality (GitHub blog, 2026) — Copilot's task-tool prompt compression, numbers and regression story above.
• DSPy — framework whose optimizers rewrite prompts against a metric; a token-count term in the metric turns it into a compressor.
• promptfoo — behavioral test runner for prompts; usable as the acceptance gate of the loop.
• APE: Large Language Models Are Human-Level Prompt Engineers (Zhou et al., 2022) — model-generated prompt candidates selected by score.
• OPRO: Large Language Models as Optimizers (Yang et al., 2023) — iterative prompt optimization with past candidates and scores in the meta-prompt.