Model Routing
Model routing sends each subtask to the cheapest or smallest model capable of handling it. Not every step requires a large reasoning model; classification, formatting, and simple edits can be done by smaller models at a fraction of the cost.
When the selection happens automatically at runtime rather than being chosen manually by the user or pinned in application configuration, this is also called auto-routing, an auto-router, or auto-switching.
Routing picks one model to run the work. When two models run the same task together — a cheap one executing, an expensive one advising — that is Fusion Routing, a separate concept.
Strategies
Task Intent Routing
Classify the incoming request by task type (e.g., "quick explanation", "focused edit", "multi-file change", "debugging") and route to the model tier optimized for that category. A small model can handle the classification itself. Different kinds of work do not all benefit from the same level of reasoning.
Cascading
Start with the cheapest model. If its output fails a quality check (confidence threshold, syntax validation, or human review), escalate to the next tier. This avoids over-provisioning for easy tasks.
Model-as-a-Judge
Use a strong model to evaluate or correct outputs from a weak model. The strong model sees only the candidate output, not the full problem context, keeping its input small.
Output-format Routing
If the task is purely structural (e.g., reformat JSON, extract fields), a small model or even deterministic code suffices. Reserve large models for open-ended reasoning.
Learned Escalation
Instead of labeling tasks as simply "easy" or "hard," train the router on where models actually diverge. For each query, score responses from a less capable and a more capable model across quality dimensions. The router learns when the stronger model adds value and when a more efficient model produces an equally good result. For context-dependent messages in longer sessions, train on complete multi-turn conversations including original user intent, recent assistant responses, and conversation metadata.
Trade-offs
| Strategy | Latency | Cost | Implementation |
|---|---|---|---|
| Task-based | Low | Low | Medium |
| Cascading | Medium | Lowest | High |
| Model-as-a-Judge | Medium | Medium | Medium |
See Also
- Cache-aware Routing — a routing strategy that preserves prompt cache efficiency.
- Reasoning Suppression — routing reasoning to a cheaper model is a routing use case.
- Fail Fast Validation — validation can be done by a cheap model before routing to an expensive one.
- Effort Calibration — routing picks which model; effort calibration picks how hard the chosen model thinks. The two combine.
- Fusion Routing — instead of picking one model, pair a cheap build model with an expensive advisor.
- Context Isolation via Subagents — the orthogonal axis: routing decides which model runs the subtask, isolation decides whose context holds the data.
Tools
- GitHub Copilot Auto model selection — combines task intent and model health to route to the best-fit model, with cache-aware switching and learned escalation.
- RouteLLM — router that learns to choose between strong and weak models based on query features.
- Martian Model Router — dynamic routing across multiple model providers.
- LiteLLM Router — load-balancing and fallback routing with cost-based priorities.
- BTL Runtime — chooses a healthy eligible provider route using capability, health, cost, and latency signals, with pre-output fallback.
- Factory Router — automatically selects a model per Droid session and can move a struggling session to a more capable model.
- Kilo Auto Model — classifies session intent and selects models from a tier-specific strategy.
- NadirClaw — self-hosted OpenAI-compatible router for automatic cheap/local versus premium model selection.
- OpenRouter Auto — automatic provider/model routing behind the
openrouter/autoendpoint. - OrcaRouter Adaptive Routing — adaptive model routing service.
- Weave Router — model router for agentic systems.