Production LLM AI gateway
Need a unified gateway in front of many model providers with fallbacks, virtual keys, budgets, guardrails, and request logs — not just a model marketplace.
At a glance — Oct 2026
Updated 2026-10-27 · forward this page: /jtbds/llm-ai-gateway#briefScores as of 2026-09-14 (43 days old; Depot marks scores stale after 30 days). The pick is still Depot's current editorial pick; it has not been re-reviewed since. Prices below carry their own dates.
- Depot Pick
- Portkey · 8.31/10 for this job (curated estimate, high confidence)
DX + reliability + capability for fallbacks/keys/guardrails. OpenRouter wins catalog breadth; Portkey wins production gateway controls.
- Runner-up
- Vercel AI Gateway · 8.21/10
Scores higher than the pick on: Price / performance, Developer & agent DX.
- When not to use the pick
- You only need a multi-model catalog/router (OpenRouter), pure cheap request logs (Helicone), or are standardized on Palo Alto Prisma AIRS enterprise gateway.
- Lock-in / migration
- The pick has lower stored lock-in risk than runner-up Vercel AI Gateway (5 vs 5.8). Pick exit signals: OpenAI-compatible API (provider tag `openai-compat`). Self-host fallback listed for this job: LiteLLM.
- Sourced price (no comparable unit)
Pick: no sourced price in this job's unit (no_matching_sourced_price).
No cheapest SKU (no_comparable_unit_for_job). No price was invented.
- Free option
- LiteLLM (oss) — source, as of 2026-10-25
- Failure modes to watch
- Non-determinism: Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.
- Observability gap: Alert on a drop in trace ingestion and on certificate expiry for the trace host. Do not treat a missing trace as a successful quiet run.
- Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
Cite: Agent Provider Depot, Depot Pick for llm-ai-gateway: portkey, scores as of 2026-09-14. Agents: GET /api/decide?jtbd=llm-ai-gateway
Free Production LLM AI gateway options
Portkey
DX + reliability + capability for fallbacks/keys/guardrails. OpenRouter wins catalog breadth; Portkey wins production gateway controls.
Pricing (directional): As of 2026-09-14 (portkey.ai/pricing): Developer free forever (10k logs/mo, 3-day log / 30-day metrics retention). Production $49/mo (100k logs, +$9/100k overage, 30/90-day retention, guardrails, RBAC). Enterprise custom (10M+ logs, VPC, HI…
Lock-in risk: higher = more lock-in (worse). Overall already inverts this dimension.
Depot pick for production AI gateway (fallbacks, virtual keys, budgets). OpenRouter when you want a model catalog/router; Helicone for request-path cost traces; Prisma AIRS when already in Palo Alto estate.
Quote for Production LLM AI gateway
Depot Pick stays Portkey. Prices below are sourced SKU rows only (as of 2026-10-27). Input and output are not blended. GET /api/quote?jtbd=llm-ai-gateway
Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null.
- Production usd_per_month $49 source as of 2026-10-03
3 of 6 candidates have no sourced price row. 6 are excluded from the Matching price comparison (none). guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored
Runners-up
Vercel AI Gateway
Best when the app already ships on Vercel/AI SDK. Portkey remains production-gateway default; OpenRouter for…
Cloudflare AI Gateway
Free edge gateway peer to Portkey/OpenRouter. Depot llm-ai-gateway pick stays Portkey for virtual…
OpenRouter
Best multi-model router for agents that need failover and experimentation. Pin providers in production; don’t…
Helicone
Maintenance mode since 2026-03-03 (Mintlify); site now says 'no longer part of Mintlify'. Runner-up only, not a pick.
Cloudflare Workers AI
Best when your agent already runs on Workers and you want colocated small/open models. Not a frontier brain.
When NOT to use the top pick
You only need a multi-model catalog/router (OpenRouter), pure cheap request logs (Helicone), or are standardized on Palo Alto Prisma AIRS enterprise gateway.
Full ranking (score table)
Ranked with this JTBD's weights. Scores are curated estimates unless marked otherwise. Agents: prefer /api/decide then this table for tie-breaks.
| # | Provider | Overall | Fit | $/perf | Latency | DX | As of |
|---|---|---|---|---|---|---|---|
| 1 | Cloudflare AI Gateway | 8.4 | 8.0 | 9.4 | 9.0 | 8.2 | 2026-09-14 |
| 2 | PortkeyDepot Pick | 8.3 | 8.8 | 8.2 | 8.6 | 9.0 | 2026-09-14 |
| 3 | Vercel AI Gateway | 8.2 | 8.2 | 9.0 | 8.6 | 9.2 | 2026-09-14 |
| 4 | Helicone | 8.1 | 8.2 | 8.5 | 8.4 | 8.8 | 2026-10-19 |
| 5 | OpenRouter | 8.0 | 8.8 | 7.8 | 7.6 | 9.2 | 2026-09-14 |
| 6 | Cloudflare Workers AI | 7.7 | 6.8 | 8.5 | 8.2 | 8.4 | 2026-09-14 |
Score methodology for this job
DX + reliability + capability for fallbacks/keys/guardrails. OpenRouter wins catalog breadth; Portkey wins production gateway controls.
capabilityFit: 25%pricePerformance: 18%latencyReliability: 15%developerDx: 15%privacyPosture: 10%lockInRisk: 7%maturity: 10%