LLM rate-limit & backoff shield

Scores as of 2026-09-14 (43 days old; Depot marks scores stale after 30 days). The pick is still Depot's current editorial pick; it has not been re-reviewed since. Prices below carry their own dates.

Depot Pick

Cloudflare AI Gateway · 8.42/10 for this job (curated estimate, high confidence)

Editorial replacement 2026-10-21: the former pick Helicone (8.12 for this job) has been in maintenance mode since 2026-03-03 (https://www.mintlify.com/blog/mintlify-acquires-helicone) and its site now says 'Helicone is no longer part of Mintlify. Details to come soon.' (https://www.helicone.ai/, re-checked 2026-10-21), so ownership and roadmap are unclear. Replaced by the highest-scoring live runner-up, Cloudflare AI Gateway (8.42 for this job; rate limiting is a free core feature, fixed or sliding windows, 429 on excess, per https://developers.cloudflare.com/ai-gateway/features/rate-limiting/ and /reference/pricing/; per user/project/team rate and budget limits with fallbacks via Dynamic Routing per /features/dynamic-routing/; all re-checked 2026-10-21; AI Gateway component 'operational' on cloudflarestatus.com 2026-10-21). Helicone stays a runner-up with the maintenance_mode flag. No score changed. Distinct from `llm-ai-gateway` (routing/guardrails). Focus rate limits + cost ceilings.

Runner-up

Portkey · 8.29/10

Scores higher than the pick on: Capability fit, Developer & agent DX.

When not to use the pick

Need per-virtual-key budgets and a full governance gateway (Portkey); Vercel already terminates AI traffic (Vercel AI Gateway); Helicone only if you accept maintenance-mode risk (since 2026-03-03, ownership unclear).

Lock-in / migration

The pick has lower stored lock-in risk than runner-up Portkey (4 vs 5).

Sourced price (no comparable unit)

Pick: no sourced price in this job's unit (no_sourced_price).

No cheapest SKU (no_comparable_unit_for_job). No price was invented.

Free option

No free resource lists this job. A cheap paid SKU is not a free option.

Failure modes to watch

• Identity confusion: Bind every tool call to a user or tenant id. Refuse the call if that id changed mid-task. For a public bot, the operator owns the words the bot shows.
• Non-determinism: Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.
• Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.

Reported outcomes (submitted reports only)

This job: loading… JSON · reports never change scores · seed/test records excluded

Depot Pick (Cloudflare AI Gateway) on this job: loading… JSON · reports never change scores · seed/test records excluded

Cite: Agent Provider Depot, Depot Pick for llm-rate-limit-shield: cloudflare-ai-gateway, scores as of 2026-09-14. Agents: GET /api/decide?jtbd=llm-rate-limit-shield