Online Bandit & Pareto Model Routing Adaptation (Cost / Latency / Quality Learning)
Continuously adapt model and provider routing weights per task class using live latency, error-rate, and verifier pass-rate telemetry — routing routine steps to fast/cheap models while escalating hard steps to frontier reasoning models.
At a glance — Oct 2026
Updated 2026-10-28 · forward this page: /jtbds/online-bandit-model-routing-adaptation#briefPromotion class: deterministic_source_reverified · 8.0 ± 0.5 (7.5–8.5 · editorial_prior) · Pricing verified 2026-10-25.
- Depot Pick
- OpenRouter · 8.04/10 for this job (curated estimate, high confidence)
OpenRouter exposes unified routing across 300+ models with real-time provider latency/uptime telemetry, automatic fallback chains, price/throughput sorting, and zero markup on token rates (5.5% credit purchase fee as of 2026-10-28), making it the cleanest actuator for adaptive multi-model routing.
- Runner-up
- Portkey · 8.36/10
Scores higher than the pick on: Price / performance, Latency / reliability, Privacy / data retention, Maturity / track record.
- When not to use the pick
- Enterprise AI gateway with semantic caching, guardrails, and virtual keys across teams — Portkey; observability-first proxy with header-based caching and cost analytics — Helicone; offline dataset Pareto scoring to set initial routing priors — Braintrust.
- Lock-in / migration
- The pick has lower stored lock-in risk than runner-up Portkey (4 vs 5). Pick exit signals: OpenAI-compatible API (provider tag `openai-compat`).
- Sourced price (usd_per_million_input_tokens)
Pick's lowest sourced SKU: anthropic/claude-sonnet-5.5 $2 (usd_per_million_input_tokens) — source, as of 2026-10-28, re-checked 2026-10-28
Cheapest matching SKU: OpenRouter anthropic/claude-sonnet-5.5 $2 — source, as of 2026-10-28, re-checked 2026-10-28. The pick is the cheapest.
Price history: first seen 2026-10-03 · last changed no change recorded · history JSON
- Free option (runnable / substitutable)
- No runnable free tier lists this job. A cheap paid SKU or benchmark dataset is not a runnable free option.
- Failure modes to watch
- Non-determinism: Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.
- Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
Cite: Agent Provider Depot, Depot Pick for online-bandit-model-routing-adaptation: openrouter (editorial prior, scores as of 2026-10-28). Agents: GET /api/decide?jtbd=online-bandit-model-routing-adaptation
All sourced prices, cheapest first
OpenRouter
OpenRouter exposes unified routing across 300+ models with real-time provider latency/uptime telemetry, automatic fallback chains, price/throughput sorting, and zero markup on token rates (5.5% credit purchase fee as of 2026-10-28), making it the cleanest actuator for adaptive multi-model routing.
Pricing (directional): As of 2026-10-28: Multi-provider LLM router with OpenAI-compatible API; pass-through model pricing plus small platform fee. Best for prototyping multi-model and failover; production should pin critical paths to direct providers.
Top pick (8.04) and #2 runner-up (8.36) are separated by 0.32 points (<= 0.35 weight-sensitivity band). Treat them as co-picks and use the workload tie-breaker rule below rather than the raw composite decimal.
Workload Tie-Breaker Rule: Enterprise AI gateway with semantic caching, guardrails, and virtual keys across teams — Portkey; observability-first proxy with header-based caching and cost analytics — Helicone; offline dataset Pareto scoring to set initial routing priors — Braintrust.
Lock-in risk: higher = more lock-in (worse). Overall already inverts this dimension.
Best multi-model router for agents that need failover and experimentation. Pin providers in production; don’t treat OpenRouter as your only SLA.
Quote for Online Bandit & Pareto Model Routing Adaptation (Cost / Latency / Quality Learning)
Depot Pick stays OpenRouter. Prices below are sourced SKU rows only (as of 2026-10-28). Input and output are not blended. GET /api/quote?jtbd=online-bandit-model-routing-adaptation
Only 1 candidate has a sourced input $/1M; nothing was compared, so pickIsPriceWinner is null.
- anthropic/claude-sonnet-5.5 usd_per_million_input_tokens $2 source as of 2026-10-28
- x-ai/grok-4.7 usd_per_million_input_tokens $2 source as of 2026-10-28
- anthropic/claude-sonnet-5.5 usd_per_million_output_tokens $10 source as of 2026-10-28
- x-ai/grok-4.7 usd_per_million_output_tokens $6 source as of 2026-10-28
2 of 4 candidates have no sourced price row. 3 are excluded from the Input $/1M comparison (chat_completion_input). min sourced input SKU among candidates; not quality-adjusted
Runners-up
Portkey
Depot pick for production AI gateway (fallbacks, virtual keys, budgets). OpenRouter when you want a model…
Helicone
Maintenance mode since 2026-03-03 (Mintlify); site now says 'no longer part of Mintlify'. Runner-up only, not a pick.
Braintrust
Depot pick when evals are the product quality gate. Pair with Helicone/Langfuse for cheap request logs if…
When NOT to use the top pick
Enterprise AI gateway with semantic caching, guardrails, and virtual keys across teams — Portkey; observability-first proxy with header-based caching and cost analytics — Helicone; offline dataset Pareto scoring to set initial routing priors — Braintrust.
Full ranking (score table)
Ranked with this JTBD's weights. Scores are curated estimates unless marked otherwise. Agents: prefer /api/decide then this table for tie-breaks.
| # | Provider | Overall | Fit | $/perf | Latency | DX | Lock-In Risk | As of |
|---|---|---|---|---|---|---|---|---|
| 1 | Portkey | 8.4 | 8.8 | 8.2 | 8.6 | 9.0 | 5.0 | 2026-10-28 |
| 2 | Braintrust | 8.3 | 9.2 | 7.5 | 8.6 | 8.8 | 5.0 | 2026-10-28 |
| 3 | Helicone | 8.2 | 8.2 | 8.5 | 8.4 | 8.8 | 4.0 | 2026-10-28 |
| 4 | OpenRouterDepot Pick | 8.0 | 8.8 | 7.8 | 7.6 | 9.2 | 4.0 | 2026-10-28 |
Score methodology for this job
Contextual bandit routing requires a trustworthy reward signal: never route purely on token price or TTFT without joining downstream task success (`verifiedSolvedCount`), or the router will collapse onto a fast model that fails silently.
capabilityFit: 24%pricePerformance: 20%latencyReliability: 20%developerDx: 16%privacyPosture: 8%lockInRisk: 6%maturity: 6%