Routing is a policy decision, so the policy surface is the product. Six strategies, ten providers, and per-key controls that let platform teams delegate access without losing the guardrails.
A strategy is a weighting over the same four scores. Set one default for the organisation and override it per API key, per route, or per tenant.
The default. Classify, then apply the weights tuned for that task type. Best all-round savings without a quality cliff.
Cheapest model that satisfies the hard constraints. For high-volume, low-stakes traffic where budget is the binding limit.
Highest benchmark score within a cost ceiling you set. For customer-facing output where a bad answer costs more than tokens.
Lowest trailing p95 among qualified models. For interactive surfaces with a response-time budget.
Run the ladder cheapest-first, escalate on a failed quality check. Deepest savings on mixed-difficulty workloads.
Force a specific model for a key, route or tenant. For regulated workloads and A/B baselines.
New models are benchmarked and priced before they are eligible for routing. Providers in a degraded state are excluded from candidate sets until they recover.
| Provider | Models in rotation | Regions | Status |
|---|---|---|---|
| OpenAI | GPT-4o, GPT-4o mini, o3-mini | US, EU | Operational |
| Anthropic | Claude Sonnet 4, Claude Haiku 3.5 | US, EU | Operational |
| Gemini 2.0 Flash, Gemini 1.5 Pro | US, EU, APAC | Operational | |
| Groq | Llama 3.3 70B, Llama 3.1 8B | US | Operational |
| Mistral | Mistral Large, Mistral Small | EU | Operational |
| xAI | Grok 3 | US | Degraded |
Everything the router decides is queryable, and everything it is allowed to decide is configurable.
Look up any request by ID and get the classification, the full candidate table, the cascade path, and the cost proof.
Monthly caps and rate limits per API key, with soft-warning thresholds and automatic downgrade instead of hard failure.
Allow-lists, per-task preferred and fallback models, and quality floors expressed as a single number your team can reason about.
CSV and JSON export, per-request webhooks, and an OpenTelemetry span for every routing decision.
