Every request passes through four stages before it reaches a provider. The whole sequence runs in-process, adds a millisecond and a half, and writes a complete record of its own reasoning.
Classification is cheap and local. Scoring is arithmetic over a pricing and quality table refreshed hourly. Nothing in the hot path calls another model.
A local classifier labels the request by task type — factual, extraction, summarisation, code, creative, analysis or math — and returns a confidence score. Below your confidence floor, the request falls back to the pinned default rather than guessing.
Models are filtered to those that satisfy the request: context length, modality, tool support, your allow-list, and current provider health. A degraded provider drops out of consideration automatically.
Each surviving candidate is scored on cost, quality, latency and context headroom, weighted by your active strategy. The highest total wins; ties break toward the cheaper model.
The request is forwarded to the winner. In cascade mode the response is checked against the quality bar and escalated if it falls short. Cost, baseline and the full decision trace are written before the response returns.
Weights are yours to set per strategy. Below is a real scoring pass for a factual lookup under the default Task-Aware weights.
| Candidate | Provider | Cost | Quality | Latency | Total | |
|---|---|---|---|---|---|---|
| Gemini 2.0 Flash | 96 | 91 | 74 | 87.4 | Selected | |
| GPT-4o mini | OpenAI | 93 | 88 | 79 | 85.1 | |
| Claude Haiku 3.5 | Anthropic | 88 | 90 | 71 | 82.9 | |
| Llama 3.3 70B | Groq | 97 | 79 | 68 | 82.0 | |
| Claude Sonnet 4 | Anthropic | 41 | 96 | 58 | 68.2 |
For workloads where a cheap model is usually enough but occasionally isn't, cascade runs the ladder in order and stops at the first response that clears the bar.
Total spend $0.0012 against a $0.0366 single-model baseline. The failed attempt is billed and shown — cascade is honest about its own overhead.
Every header is also written to the decision log, so finance can reconcile a month of spend without instrumenting your application.
