How do agents decide without a human?

GET /api/decide?jtbd=edge-llm-inference returns pick, runnersUp, cite, and nextActions.

← All jobs

Edge-colocated LLM inference

Q: What is the Depot Pick for Edge-colocated LLM inference?

Cloudflare Workers AI. Colocation + pricePerf. Workers AI for edge; Groq for raw speed off-edge.

Agent runs on edge/Workers and needs low-latency small/open models next to the runtime.

GET /api/decide?jtbd=edge-llm-inference API embed Cloudflare Workers AI vs Groq Cloudflare Workers AI vs Fireworks AI

At a glance — Oct 2026

Updated 2026-10-27 · forward this page: /jtbds/edge-llm-inference#brief

Scores as of 2026-09-14 (43 days old; Depot marks scores stale after 30 days). The pick is still Depot's current editorial pick; it has not been re-reviewed since. Prices below carry their own dates.

Depot Pick

Cloudflare Workers AI · 7.69/10 for this job (curated estimate, high confidence)

Colocation + pricePerf. Workers AI for edge; Groq for raw speed off-edge.

Runner-up

Groq · 7.95/10

Scores higher than the pick on: Capability fit, Price / performance, Developer & agent DX.

When not to use the pick

Need frontier reasoning quality — use Anthropic/OpenAI and accept hop latency.

Lock-in / migration

The pick has lower stored lock-in risk than runner-up Groq (4.5 vs 5). Pick exit signals: OpenAI-compatible API (provider tag `openai-compat`). Self-host fallback listed for this job: llama.cpp.

Sourced price (usd_per_million_input_tokens)

Pick's lowest sourced SKU: @cf/ibm-granite/granite-4.0-h-micro $0.017 (usd_per_million_input_tokens) — source, as of 2026-10-03, re-checked 2026-10-27

Cheapest matching SKU: Cloudflare Workers AI @cf/ibm-granite/granite-4.0-h-micro $0.017 — source, as of 2026-10-03, re-checked 2026-10-27. The pick is the cheapest.

Price history: first seen 2026-10-03 · last changed no change recorded · history JSON

Free option

Cloudflare Workers AI daily neurons (free) — source, as of 2026-10-25
llama.cpp (oss) — source, as of 2026-10-25

Failure modes to watch

Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.

Reported outcomes (submitted reports only)

This job: loading… JSON · reports never change scores · seed/test records excluded

Depot Pick (Cloudflare Workers AI) on this job: loading… JSON · reports never change scores · seed/test records excluded

Same-job, same-unit comparisons: Cloudflare Workers AI vs Groq · Cloudflare Workers AI vs OpenRouter

Cite: Agent Provider Depot, Depot Pick for edge-llm-inference: cloudflare-workers-ai, scores as of 2026-09-14. Agents: GET /api/decide?jtbd=edge-llm-inference

Free Edge-colocated LLM inference options · All sourced prices, cheapest first

Depot Pick

Cloudflare Workers AI

Colocation + pricePerf. Workers AI for edge; Groq for raw speed off-edge.

Pricing (directional): As of 2026-10-24 (developers.cloudflare.com/workers-ai/platform/pricing, fetched 2026-10-24): '$0.011 per 1,000 Neurons' with '10,000 Neurons per day at no charge' on both Workers Free and Workers Paid ('All limits reset daily at 00:00 UTC.…

Cite: Depot Pick for edge-llm-inference: cloudflare-workers-ai

Overall

7.7

curated_estimate

confidence: high · as of 2026-09-14

Capability fit

6.8

Price / performance

8.5

Latency / reliability

8.2

Developer & agent DX

8.4

Privacy / data retention

8.0

Lock-in risk (higher = worse)

4.5

Maturity / track record

7.8

Lock-in risk: higher = more lock-in (worse). Overall already inverts this dimension.

Sources: Workers AI pricing: https://developers.cloudflare.com/workers-ai/platform/pricing/ · Workers AI: https://developers.cloudflare.com/workers-ai/ · Cloudflare Workers AI product: https://www.cloudflare.com/products/workers-ai/

Best when your agent already runs on Workers and you want colocated small/open models. Not a frontier brain.

Quote for Edge-colocated LLM inference

Depot Pick stays Cloudflare Workers AI. Prices below are sourced SKU rows only (as of 2026-10-27). Input and output are not blended. GET /api/quote?jtbd=edge-llm-inference

Pick is the cheapest sourced input $/1M among candidates (0.017).

@cf/ai4bharat/indictrans2-en-indic-1B usd_per_million_input_tokens $0.342 source as of 2026-10-03
@cf/aisingapore/gemma-sea-lion-v4-27b-it usd_per_million_input_tokens $0.351 source as of 2026-10-03
@cf/baai/bge-base-en-v1.5 usd_per_million_input_tokens $0.067 source as of 2026-10-03
@cf/baai/bge-large-en-v1.5 usd_per_million_input_tokens $0.204 source as of 2026-10-03
@cf/baai/bge-m3 usd_per_million_input_tokens $0.012 source as of 2026-10-03
@cf/baai/bge-reranker-base usd_per_million_input_tokens $0.003 source as of 2026-10-03
@cf/baai/bge-small-en-v1.5 usd_per_million_input_tokens $0.02 source as of 2026-10-03
@cf/cloudflare/clef-flash usd_per_million_input_tokens $0.038 source as of 2026-10-19
@cf/cloudflare/clef usd_per_million_input_tokens $0.24 source as of 2026-10-03
@cf/deepseek-ai/deepseek-r1-distill-qwen-32b usd_per_million_input_tokens $0.497 source as of 2026-10-03
@cf/deepseek-ai/deepseek-v4-flash-0731 usd_per_million_input_tokens $0.44 source as of 2026-10-03
@cf/deepseek-ai/deepseek-v4-pro-0813 usd_per_million_input_tokens $1.32 source as of 2026-10-03
@cf/google/gemma-3-12b-it usd_per_million_input_tokens $0.345 source as of 2026-10-03
@cf/google/gemma-4-26b-a4b-it usd_per_million_input_tokens $0.1 source as of 2026-10-03
@cf/huggingface/distilbert-sst-2-int8 usd_per_million_input_tokens $0.026 source as of 2026-10-03
@cf/ibm-granite/granite-4.0-h-micro usd_per_million_input_tokens $0.017 source as of 2026-10-03
@cf/meta/llama-2-7b-chat-fp16 usd_per_million_input_tokens $0.556 source as of 2026-10-03
@cf/meta/llama-3.1-70b-instruct-fp8-fast usd_per_million_input_tokens $0.293 source as of 2026-10-03
@cf/meta/llama-3.1-8b-instruct-awq usd_per_million_input_tokens $0.123 source as of 2026-10-03
@cf/meta/llama-3.1-8b-instruct-fp8-fast usd_per_million_input_tokens $0.045 source as of 2026-10-03
@cf/meta/llama-3.1-8b-instruct-fp8 usd_per_million_input_tokens $0.152 source as of 2026-10-03
@cf/meta/llama-3.1-8b-instruct usd_per_million_input_tokens $0.282 source as of 2026-10-03
@cf/meta/llama-3.2-11b-vision-instruct usd_per_million_input_tokens $0.049 source as of 2026-10-03
@cf/meta/llama-3.2-1b-instruct usd_per_million_input_tokens $0.027 source as of 2026-10-03
@cf/meta/llama-3.2-3b-instruct usd_per_million_input_tokens $0.051 source as of 2026-10-03
@cf/meta/llama-3.3-70b-instruct-fp8-fast usd_per_million_input_tokens $0.293 source as of 2026-10-03
@cf/meta/llama-3-8b-instruct-awq usd_per_million_input_tokens $0.123 source as of 2026-10-03
@cf/meta/llama-3-8b-instruct usd_per_million_input_tokens $0.282 source as of 2026-10-03
@cf/meta/llama-4-scout-17b-16e-instruct usd_per_million_input_tokens $0.27 source as of 2026-10-03
@cf/meta/llama-guard-3-8b usd_per_million_input_tokens $0.484 source as of 2026-10-03
@cf/meta/m2m100-1.2b usd_per_million_input_tokens $0.342 source as of 2026-10-03
@cf/mistral/mistral-7b-instruct-v0.1 usd_per_million_input_tokens $0.11 source as of 2026-10-03
@cf/mistralai/mistral-small-3.1-24b-instruct usd_per_million_input_tokens $0.351 source as of 2026-10-03
@cf/moondream/moondream3.1-9B-A2B usd_per_million_input_tokens $0.3 source as of 2026-10-03
@cf/moonshotai/kimi-k2.5 usd_per_million_input_tokens $0.6 source as of 2026-10-03
@cf/moonshotai/kimi-k2.6 usd_per_million_input_tokens $0.95 source as of 2026-10-03
@cf/moonshotai/kimi-k2.7-code usd_per_million_input_tokens $0.95 source as of 2026-10-03
@cf/nvidia/nemotron-3-120b-a12b usd_per_million_input_tokens $0.5 source as of 2026-10-03
@cf/openai/gpt-oss-120b usd_per_million_input_tokens $0.35 source as of 2026-10-03
@cf/openai/gpt-oss-20b usd_per_million_input_tokens $0.2 source as of 2026-10-03
@cf/pfnet/plamo-embedding-1b usd_per_million_input_tokens $0.019 source as of 2026-10-03
@cf/qwen/qwen2.5-coder-32b-instruct usd_per_million_input_tokens $0.66 source as of 2026-10-03
@cf/qwen/qwen3-30b-a3b-fp8 usd_per_million_input_tokens $0.051 source as of 2026-10-03
@cf/qwen/qwen3.8-27b usd_per_million_input_tokens $0.45 source as of 2026-10-03
@cf/qwen/qwen3-embedding-0.6b usd_per_million_input_tokens $0.012 source as of 2026-10-03
@cf/qwen/qwq-32b usd_per_million_input_tokens $0.66 source as of 2026-10-03
@cf/zai-org/glm-4.7-flash usd_per_million_input_tokens $0.06 source as of 2026-10-03
@cf/zai-org/glm-5.2 usd_per_million_input_tokens $1.4 source as of 2026-10-03
@cf/zai-org/glm-5.3-flash usd_per_million_input_tokens $0.15 source as of 2026-10-03
@cf/zai-org/glm-5.3 usd_per_million_input_tokens $1.4 source as of 2026-10-03
@cf/ai4bharat/indictrans2-en-indic-1B usd_per_million_output_tokens $0.342 source as of 2026-10-03
@cf/aisingapore/gemma-sea-lion-v4-27b-it usd_per_million_output_tokens $0.555 source as of 2026-10-03
@cf/deepseek-ai/deepseek-r1-distill-qwen-32b usd_per_million_output_tokens $4.881 source as of 2026-10-03
@cf/deepseek-ai/deepseek-v4-flash-0731 usd_per_million_output_tokens $1.32 source as of 2026-10-03
@cf/deepseek-ai/deepseek-v4-pro-0813 usd_per_million_output_tokens $3.96 source as of 2026-10-03
@cf/google/gemma-3-12b-it usd_per_million_output_tokens $0.556 source as of 2026-10-03
@cf/google/gemma-4-26b-a4b-it usd_per_million_output_tokens $0.3 source as of 2026-10-03
@cf/ibm-granite/granite-4.0-h-micro usd_per_million_output_tokens $0.112 source as of 2026-10-03
@cf/meta/llama-2-7b-chat-fp16 usd_per_million_output_tokens $6.667 source as of 2026-10-03
@cf/meta/llama-3.1-70b-instruct-fp8-fast usd_per_million_output_tokens $2.253 source as of 2026-10-03
@cf/meta/llama-3.1-8b-instruct-awq usd_per_million_output_tokens $0.266 source as of 2026-10-03
@cf/meta/llama-3.1-8b-instruct-fp8-fast usd_per_million_output_tokens $0.384 source as of 2026-10-03
@cf/meta/llama-3.1-8b-instruct-fp8 usd_per_million_output_tokens $0.287 source as of 2026-10-03
@cf/meta/llama-3.1-8b-instruct usd_per_million_output_tokens $0.827 source as of 2026-10-03
@cf/meta/llama-3.2-11b-vision-instruct usd_per_million_output_tokens $0.676 source as of 2026-10-03
@cf/meta/llama-3.2-1b-instruct usd_per_million_output_tokens $0.201 source as of 2026-10-03
@cf/meta/llama-3.2-3b-instruct usd_per_million_output_tokens $0.335 source as of 2026-10-03
@cf/meta/llama-3.3-70b-instruct-fp8-fast usd_per_million_output_tokens $2.253 source as of 2026-10-03
@cf/meta/llama-3-8b-instruct-awq usd_per_million_output_tokens $0.266 source as of 2026-10-03
@cf/meta/llama-3-8b-instruct usd_per_million_output_tokens $0.827 source as of 2026-10-03
@cf/meta/llama-4-scout-17b-16e-instruct usd_per_million_output_tokens $0.85 source as of 2026-10-03
@cf/meta/llama-guard-3-8b usd_per_million_output_tokens $0.03 source as of 2026-10-03
@cf/meta/m2m100-1.2b usd_per_million_output_tokens $0.342 source as of 2026-10-03
@cf/mistral/mistral-7b-instruct-v0.1 usd_per_million_output_tokens $0.19 source as of 2026-10-03
@cf/mistralai/mistral-small-3.1-24b-instruct usd_per_million_output_tokens $0.555 source as of 2026-10-03
@cf/moondream/moondream3.1-9B-A2B usd_per_million_output_tokens $1 source as of 2026-10-03
@cf/moonshotai/kimi-k2.5 usd_per_million_output_tokens $3 source as of 2026-10-03
@cf/moonshotai/kimi-k2.6 usd_per_million_output_tokens $4 source as of 2026-10-03
@cf/moonshotai/kimi-k2.7-code usd_per_million_output_tokens $4 source as of 2026-10-03
@cf/nvidia/nemotron-3-120b-a12b usd_per_million_output_tokens $1.5 source as of 2026-10-03
@cf/openai/gpt-oss-120b usd_per_million_output_tokens $0.75 source as of 2026-10-03
@cf/openai/gpt-oss-20b usd_per_million_output_tokens $0.3 source as of 2026-10-03
@cf/qwen/qwen2.5-coder-32b-instruct usd_per_million_output_tokens $1 source as of 2026-10-03
@cf/qwen/qwen3-30b-a3b-fp8 usd_per_million_output_tokens $0.335 source as of 2026-10-03
@cf/qwen/qwen3.8-27b usd_per_million_output_tokens $3.2 source as of 2026-10-03
@cf/qwen/qwq-32b usd_per_million_output_tokens $1 source as of 2026-10-03
@cf/zai-org/glm-4.7-flash usd_per_million_output_tokens $0.4 source as of 2026-10-03
@cf/zai-org/glm-5.2 usd_per_million_output_tokens $4.4 source as of 2026-10-03
@cf/zai-org/glm-5.3-flash usd_per_million_output_tokens $0.5 source as of 2026-10-03
@cf/zai-org/glm-5.3 usd_per_million_output_tokens $4.4 source as of 2026-10-03
@cf/black-forest-labs/flux-1-schnell 512x512 tile usd_non_token $0.0000528 source as of 2026-10-03
@cf/black-forest-labs/flux-1-schnell step usd_non_token $0.0001056 source as of 2026-10-03
@cf/black-forest-labs/flux-2-dev input 512x512 tile, per step usd_non_token $0.00021 source as of 2026-10-03
@cf/black-forest-labs/flux-2-dev output 512x512 tile, per step usd_non_token $0.00041 source as of 2026-10-03
@cf/black-forest-labs/flux-2-klein-4b input 512x512 tile usd_non_token $0.000059 source as of 2026-10-03
@cf/black-forest-labs/flux-2-klein-4b output 512x512 tile usd_non_token $0.000287 source as of 2026-10-03
@cf/black-forest-labs/flux-2-klein-9b first MP (1024x1024) usd_non_token $0.015 source as of 2026-10-03
@cf/black-forest-labs/flux-2-klein-9b input image MP usd_non_token $0.002 source as of 2026-10-03
@cf/black-forest-labs/flux-2-klein-9b subsequent MP usd_non_token $0.002 source as of 2026-10-03
@cf/deepgram/aura-1 1k characters input usd_per_1k_characters $0.015 source as of 2026-10-03
@cf/deepgram/aura-2-en 1k characters input usd_per_1k_characters $0.03 source as of 2026-10-03
@cf/deepgram/aura-2-es 1k characters input usd_per_1k_characters $0.03 source as of 2026-10-03
@cf/deepgram/flux audio minute usd_per_minute $0.0077 source as of 2026-10-03
@cf/deepgram/nova-3 audio minute input usd_per_minute $0.0052 source as of 2026-10-03
@cf/deepgram/nova-3 (WebSocket) audio minute input usd_per_minute $0.0092 source as of 2026-10-03
@cf/deepseek-ai/deepseek-v4-flash-0731 usd_per_million_cached_input_tokens $0.014 source as of 2026-10-03
@cf/deepseek-ai/deepseek-v4-pro-0813 usd_per_million_cached_input_tokens $0.044 source as of 2026-10-03
@cf/leonardo/lucid-origin 512x512 tile usd_non_token $0.006996 source as of 2026-10-03
@cf/leonardo/lucid-origin step usd_non_token $0.000132 source as of 2026-10-03
@cf/leonardo/phoenix-1.0 512x512 tile usd_non_token $0.00583 source as of 2026-10-03
@cf/leonardo/phoenix-1.0 step usd_non_token $0.00011 source as of 2026-10-03
@cf/microsoft/resnet-50 usd_per_million_images $2.51 source as of 2026-10-03
@cf/moonshotai/kimi-k2.5 usd_per_million_cached_input_tokens $0.1 source as of 2026-10-03
@cf/moonshotai/kimi-k2.6 usd_per_million_cached_input_tokens $0.16 source as of 2026-10-03
@cf/moonshotai/kimi-k2.7-code usd_per_million_cached_input_tokens $0.19 source as of 2026-10-03
@cf/myshell-ai/melotts audio minute usd_per_minute $0.0002 source as of 2026-10-03
@cf/openai/whisper audio minute usd_per_minute $0.0005 source as of 2026-10-03
@cf/openai/whisper-large-v3-turbo audio minute usd_per_minute $0.0005 source as of 2026-10-03
@cf/pipecat-ai/smart-turn-v2 audio minute input usd_per_minute $0.00033795 source as of 2026-10-03
@cf/zai-org/glm-5.2 usd_per_million_cached_input_tokens $0.26 source as of 2026-10-03
@cf/zai-org/glm-5.3 usd_per_million_cached_input_tokens $0.26 source as of 2026-10-03
@cf/zai-org/glm-5.3-flash usd_per_million_cached_input_tokens $0.03 source as of 2026-10-03
Workers Paid neurons usd_per_1k_neurons $0.011 source as of 2026-10-03

0 of 4 candidates have no sourced price row. 1 are excluded from the Input $/1M comparison (chat_completion_input). min sourced input SKU among candidates; not quality-adjusted

Runners-up

Groq

Use when latency is the product. Do not build your whole company on one Groq model SKU — they rotate.

8.0

llmlpulatencyopen-models

Fireworks AI

Top pick for production open-model serving with real DX. Prefer over random GPU DIY when you want tokens not…

8.3

llmopen-modelsinferencefinetune

OpenRouter

Best multi-model router for agents that need failover and experimentation. Pin providers in production; don’t…

8.0

llmroutermulti-provideropenai-compat

When NOT to use the top pick

Need frontier reasoning quality — use Anthropic/OpenAI and accept hop latency.

Full ranking (score table)

Ranked with this JTBD's weights. Scores are curated estimates unless marked otherwise. Agents: prefer /api/decide then this table for tie-breaks.

#	Provider	Overall	Fit	$/perf	Latency	DX	As of
1	Fireworks AI	8.3	8.6	8.8	8.4	9.0	2026-09-14
2	OpenRouter	8.0	8.8	7.8	7.6	9.2	2026-09-14
3	Groq	8.0	7.8	9.0	8.2	8.8	2026-09-14
4	Cloudflare Workers AIDepot Pick	7.7	6.8	8.5	8.2	8.4	2026-09-14

Score methodology for this job

Colocation + pricePerf. Workers AI for edge; Groq for raw speed off-edge.

capabilityFit: 25%
pricePerformance: 18%
latencyReliability: 15%
developerDx: 15%
privacyPosture: 10%
lockInRisk: 7%
maturity: 10%