Effort Calibration

Some model APIs expose a knob for how much internal deliberation a request gets (effort, reasoning_effort, thinking-token budget). Effort Calibration is choosing that level per task from measured cost-vs-accuracy data instead of defaulting every request to maximum effort.

How It Works

Cost and accuracy scale non-linearly with effort, and the curve differs by model. A stronger model at low effort can beat a weaker model at high effort for less money, so the calibration question is not "how much effort does this model support" but "what is the cheapest effort level, on any available model, that clears the accuracy bar for this task."

What Changes vs. Always-Max

• Measure accuracy and cost at each discrete effort level on a held-out set before picking a default.
• Prefer a stronger model at low effort over a weaker model at high effort when both are on the table — check, don't assume the weaker model is cheaper.
• Treat effort as a per-task-type setting, not a global constant: classification and formatting calls rarely need max effort; multi-step debugging often does.
• Re-measure after a model upgrade — the cost/accuracy curve shifts with the model, so a previously-tuned effort level can be stale.

Limitations

• Changing effort mid-conversation invalidates the prompt cache on most providers, so calibration should happen at session start, not turn-by-turn.
• Requires an evaluation set representative enough that the measured curve transfers to production traffic.
• Discrete effort levels (not continuous), so the achievable cost/accuracy points are a fixed small set per model.

See Also

• Model Routing — routing picks which model; effort calibration picks how hard the chosen model thinks. The two combine: route to a model, then calibrate its effort.
• Reasoning Suppression — suppression hides or offloads chain-of-thought; calibration controls how much reasoning is generated in the first place.
• Prompt Caching — effort changes typically break the cache, so calibrate once per session rather than per turn.
• Fusion Routing — the coarse version: instead of raising effort on the running model, ask a stronger one.

Tools

• Anthropic effort parameter — discrete effort levels with documented non-linear cost/accuracy tradeoffs.
• OpenAI reasoning_effort parameter — controls depth of internal reasoning per request.
• /claude-api hillclimb (Claude Code) — searches model/effort/prompt combinations against a train/test eval split to find a Pareto-optimal configuration.