{"jtbd":{"id":"ultra-low-latency","slug":"ultra-low-latency","name":"Ultra-low-latency token streaming","shortName":"Ultra-low-latency token streaming","problemStatement":"Voice/realtime UX needs extremely fast tokens/sec on supported models."},"pick":{"id":"groq","slug":"groq","name":"Groq","tagline":"Use when latency is the product. Do not build your whole company on one Groq model SKU — they rotate.","website":"https://groq.com","overall":8.05,"scoreBand":{"low":7.5,"mid":8.1,"high":8.7,"halfWidth":0.6,"basis":"editorial_prior","display":"7.5–8.7 (editorial prior)"},"scoreLayers":{"editorialPrior":8.05,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"Per-token with free tier; LPU inference; narrow production model set (Llama/Qwen/GPT-OSS etc.); batch discounts. Llama Guard 4 12B (model id meta-llama/llama-guard-4-12b) is still documented at console.groq.com/docs/model/meta-llama/llama-guard-4-12b as an open safety classifier; its per-token price figure was removed because it couldn't be re-sourced on 2026-10-21 (no price on that page or on console.groq.com/docs/models as of 2026-10-21). See the stored rows for sourced Groq prices, including the Llama Prompt Guard 2 classifiers.","scores":{"capabilityFit":7.8,"pricePerformance":9,"latencyReliability":8.2,"developerDx":8.8,"privacyPosture":7,"lockInRisk":5,"maturity":7.8,"overall":8.05,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}},"runnersUp":[{"id":"cerebras","slug":"cerebras","name":"Cerebras","tagline":"Best ultra-throughput play when your model is on Cerebras (official token matrix now pinned). Pair with…","website":"https://www.cerebras.ai","overall":7.62,"scoreBand":{"low":7,"mid":7.6,"high":8.2,"halfWidth":0.6,"basis":"editorial_prior","display":"7.0–8.2 (editorial prior)"},"scoreLayers":{"editorialPrior":7.62,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"As of 2026-09-14: Official public models API (api.cerebras.ai/public/v1/models, HuggingFace format) pins GPT-OSS-120B $0.35/$0.75 per 1M in/out (131k ctx) and Qwen3.8-27B $0.99/$1.49 (65k ctx). Marketing cerebras.ai/inference still shows $5 free credit; Developer PayGo from $10 + 10× rate limits; Enterprise dedicated. Official docs.cerebras.ai public-models endpoint is the token matrix of record — not PricePerToken.","scores":{"capabilityFit":8,"pricePerformance":8.2,"latencyReliability":7.8,"developerDx":7.5,"privacyPosture":7.2,"lockInRisk":5.5,"maturity":7.5,"overall":7.62,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}},{"id":"sambanova","slug":"sambanova","name":"SambaNova","tagline":"Public token matrix now exists — gpt-oss-120b $0.22/$0.59 is the value SKU. Indie agents still default…","website":"https://sambanova.ai","overall":7.65,"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.65,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"As of 2026-09-14 (cloud.sambanova.ai/plans/pricing): MiniMax-M2.7 $0.60/$2.40 per 1M (cached in $0.06); DeepSeek V3.1/V3.2 $3/$4.50; gemma-4-31B-it $0.38/$1.15; gpt-oss-120b $0.22/$0.59; Llama 3.3 70B $0.60/$1.20; MiniMax-M3 $0.60/$2.40. RDU silicon, OpenAI-compat. Enterprise plans still sales-assisted.","scores":{"capabilityFit":8,"pricePerformance":7.8,"latencyReliability":8.2,"developerDx":6.8,"privacyPosture":8,"lockInRisk":6,"maturity":8,"overall":7.65,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}},{"id":"fireworks","slug":"fireworks","name":"Fireworks AI","tagline":"Top pick for production open-model serving with real DX. Prefer over random GPU DIY when you want tokens not…","website":"https://fireworks.ai","overall":8.4,"scoreBand":{"low":7.8,"mid":8.4,"high":9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.8–9.0 (editorial prior)"},"scoreLayers":{"editorialPrior":8.4,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"Serverless per-token + on-demand GPU; day-0 OSS model support; post-training stack. Free credits often available.","scores":{"capabilityFit":8.6,"pricePerformance":8.8,"latencyReliability":8.4,"developerDx":9,"privacyPosture":7.5,"lockInRisk":3.5,"maturity":8,"overall":8.4,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}},{"id":"friendli","slug":"friendli","name":"FriendliAI","tagline":"Open-model production peer when latency/tool-calling matter. Fireworks remains depot open-model default…","website":"https://friendli.ai","overall":8.12,"scoreBand":{"low":7.5,"mid":8.1,"high":8.7,"halfWidth":0.6,"basis":"editorial_prior","display":"7.5–8.7 (editorial prior)"},"scoreLayers":{"editorialPrior":8.12,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-01","pricingNotes":"As of 2026-10-01 (friendli.ai/pricing live): Dedicated Endpoints on-demand step-up **is live** — A100 **$4.0**/hr, H100 **$5.0**/hr, H200 **$7.0**/hr, B200 **$9.0**/hr, B300 $12.0/hr (was through-Sep-30: A100 $2.9, H100 $3.9, H200 $4.5, B200 $8.9, B300 $12.0). Billed per second; no idle/startup charge. Model API token examples on page (GLM/DeepSeek/MiniMax/whisper). SOC2 + HIPAA.","scores":{"capabilityFit":8.4,"pricePerformance":8,"latencyReliability":8.4,"developerDx":8.2,"privacyPosture":8.2,"lockInRisk":4,"maturity":7.6,"overall":8.12,"kind":"curated_estimate","confidence":"high","asOf":"2026-10-01"}}],"scoreBand":{"low":7.5,"mid":8.1,"high":8.7,"halfWidth":0.6,"basis":"editorial_prior","display":"7.5–8.7 (editorial prior)"},"scoreLayers":{"editorialPrior":8.05,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"whyPickOverHigherScore":{"pickId":"groq","pickName":"Groq","pickScore":8.05,"highestScoredCandidateId":"fireworks","highestScoredCandidateName":"Fireworks AI","highestScore":8.4,"pickHasLowerCompositeScore":true,"delta":-0.35,"pickLeadsOnDimensions":["pricePerformance (9 vs 8.8)"],"explanation":"Groq (8.05) is the editorial pick for Ultra-low-latency token streaming over higher-composite Fireworks AI (8.4) because composite scores are provider-wide priors while Groq wins on pricePerformance (9 vs 8.8): reliability of latency + pricePerf. Model coverage is the hard constraint."},"cheapest":{"providerId":"groq","providerName":"Groq","skuId":"groq-gpt-oss-20b-input","item":"openai/gpt-oss-20b","plan":"openai/gpt-oss-20b — column header PRICE PER 1M TOKENS","unit":"usd_per_million_input_tokens","amountUsd":0.075,"isFloorPrice":true,"floorWarning":"Provider-wide floor SKU; may not be the flagship model scored for this JTBD.","skuCountInUnit":5,"isDepotPick":true,"sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","verifiedAsOf":"2026-10-27"},"recommendedWorkloadQuote":{"providerId":"groq","providerName":"Groq","skuId":"groq-llama-prompt-guard-2-22m-input","item":"meta-llama/llama-prompt-guard-2-22m","plan":"meta-llama/llama-prompt-guard-2-22m — column header PRICE PER 1M TOKENS","unit":"usd_per_million_input_tokens","amountUsd":0.03,"outputSkuId":"groq-llama-prompt-guard-2-22m-output","outputItem":"meta-llama/llama-prompt-guard-2-22m","outputAmountUsd":0.03,"outputUnit":"usd_per_million_output_tokens","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","pricingLastVerifiedAt":"2026-10-27","isFlagshipForJob":false,"differsFromFloorCheapest":false,"representativeQuote":{"skuId":"groq-llama-prompt-guard-2-22m-input","item":"meta-llama/llama-prompt-guard-2-22m","plan":"meta-llama/llama-prompt-guard-2-22m — column header PRICE PER 1M TOKENS","amountUsd":0.03,"inputOrUnitUsd":0.03,"unit":"usd_per_million_input_tokens","outputUsd":0.03,"outputUnit":"usd_per_million_output_tokens","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03"},"estimatedTaskCost":{"tasks":1000,"retryAndToolOverheadMultiplier":1.15,"retryToolOverheadMultiplier":1.15,"inputTokensPerTask":4000,"outputTokensPerTask":1000,"baseCostPer1kTasksUsd":0.15,"totalUsd":0.1725,"estimatedTotalUsd":0.1725,"perTaskUsd":0.000172,"estimatedCostPerTaskUsd":0.000172,"formula":"((4,000 in * $0.03/1M + 1,000 out * $0.03/1M) * 1,000 tasks) * 1.15 overhead","assumptions":"1,000 agent tasks @ 4k input + 1k output tokens/task with 1.15x retry/tool-call overhead."}},"representativeQuote":{"skuId":"groq-llama-prompt-guard-2-22m-input","item":"meta-llama/llama-prompt-guard-2-22m","plan":"meta-llama/llama-prompt-guard-2-22m — column header PRICE PER 1M TOKENS","amountUsd":0.03,"inputOrUnitUsd":0.03,"unit":"usd_per_million_input_tokens","outputUsd":0.03,"outputUnit":"usd_per_million_output_tokens","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03"},"estimatedTaskCost":{"tasks":1000,"retryAndToolOverheadMultiplier":1.15,"retryToolOverheadMultiplier":1.15,"inputTokensPerTask":4000,"outputTokensPerTask":1000,"baseCostPer1kTasksUsd":0.15,"totalUsd":0.1725,"estimatedTotalUsd":0.1725,"perTaskUsd":0.000172,"estimatedCostPerTaskUsd":0.000172,"formula":"((4,000 in * $0.03/1M + 1,000 out * $0.03/1M) * 1,000 tasks) * 1.15 overhead","assumptions":"1,000 agent tasks @ 4k input + 1k output tokens/task with 1.15x retry/tool-call overhead."},"freeAlternatives":[{"id":"cerebras-free-trial","name":"Cerebras free trial credits","kind":"free_inference","providerId":"cerebras","jtbdIds":["ultra-low-latency","cheap-volume-llm"],"whatYouGet":"$5 in free credits after adding a verified payment method; they expire 30 days after they're granted and can be used across all public models (Shared Inference). When they run out or expire, API and Playground access stop until you purchase credits.","cost":"credits_not_free_forever","limits":{"creditsUsd":5,"expiresDaysAfterGrant":30,"requiresVerifiedPaymentMethod":true,"models":{"gpt-oss-120b":{"rpm":5,"uncachedTpm":30000,"totalTpm":90000,"tph":1000000,"tpd":1000000},"qwen-3.8-27b":{"rpm":5,"uncachedTpm":30000,"totalTpm":90000,"tph":1000000,"tpd":1000000}}},"sourceUrl":"https://inference-docs.cerebras.ai/support/rate-limits","notes":"Rate-limits page fetched twice 2026-10-24 about 10 s apart (identical HTML; the page shows no last-updated date); the page's markdown view (rate-limits.md) was also fetched twice (identical) and its Free Trial table has the same values. Quotes below match the HTML text (curly apostrophes shown straight). Free Trial table (HTML text): 'gpt-oss-120b 5 30K 90K 1M 1M' and 'qwen-3.8-27b 1 5 30K 90K 1M 1M' (the second '1' is the image-limit footnote marker; columns RPM, Uncached TPM, Total TPM, TPH, TPD); K/M suffixes expanded (30K=30000, 90K=90000, 1M=1000000). The table lists exactly these two models. qwen-3.8-27b footnote: 'Image limits: 2 per request, 10 MiB total request payload'. The page says: 'New accounts receive $5 in free credits after adding a verified payment method. These credits expire 30 days after they're granted and can be used across all public models.' 'If you skip adding a payment method at sign-up, Playground and API access remain inactive until you do.' 'Is there a permanently free tier? No.' 'Cerebras doesn't currently offer a no-cost tier that renews automatically or a per-model always-free allowance.' 'This provides an overview of general limits, though specific cases may vary.' 'Rate limits apply at the organization level, not the user level, and vary based on the model.' Developer (Pay as You Go) values are not stored on this free row (a paid tier); since v0.12.129 they are in the plan-limits table (data/limits.json, cerebras / Developer (Pay as You Go)). All stored values re-confirmed unchanged on 2026-10-24. Not a Depot price-sort input.","license":null,"asOf":"2026-10-24","freeType":"free_inference","substitutable":true},{"id":"groq-free-plan-chat","name":"Groq Free Plan chat models","kind":"free_inference","providerId":"groq","jtbdIds":["cheap-volume-llm","ultra-low-latency","open-model-inference"],"whatYouGet":"Organization-level Free Plan rate limits for the chat and classifier model ids in the page payload key freeRows. Not the Developer plan table.","cost":"free","limits":{"scope":"organization","models":{"openai/gpt-oss-120b":{"rpm":30,"rpd":1000,"tpm":8000,"tpd":200000},"openai/gpt-oss-20b":{"rpm":30,"rpd":1000,"tpm":8000,"tpd":200000},"openai/gpt-oss-safeguard-20b":{"rpm":3,"rpd":1000,"tpm":2000,"tpd":200000},"qwen/qwen3.8-27b":{"rpm":30,"rpd":1000,"tpm":8000,"tpd":200000},"meta-llama/llama-prompt-guard-2-22m":{"rpm":30,"rpd":14400,"tpm":15000,"tpd":500000},"meta-llama/llama-prompt-guard-2-86m":{"rpm":30,"rpd":14400,"tpm":15000,"tpd":500000}}},"sourceUrl":"https://console.groq.com/docs/rate-limits","notes":"Rate-limits page fetched twice 2026-10-24 about 10 s apart (identical HTML; the page shows no last-updated date). Parsed from the HTML payload key freeRows (the 'Free Plan Limits' tab), which is separate from devRows (Developer plan, not stored). K suffixes expanded (1K=1000, 2K=2000, 14.4K=14400). The page says: 'The following is a high level summary and there may be exceptions to these limits.' and 'You can view the current, exact rate limits for your organization on the limits page in your account settings.' 'Rate limits apply at the organization level, not individual users.' 'Cached tokens do not count towards your rate limits.' The page also says 'some organizations are also subject to separate per-minute limits on input tokens (ITPM) and output tokens (OTPM)'; no ITPM/OTPM values are published, so none are stored. Change on 2026-10-24: freeRows now reads 'openai/gpt-oss-safeguard-20b 3 1K 2K 200K' (RPM 3, TPM 2K); before 0.12.98 this row stored RPM 30 and TPM 8000 for it. The other five model ids and their values are unchanged. No Llama chat model is in freeRows.","license":null,"asOf":"2026-10-24","freeType":"free_inference","substitutable":true}],"referenceDocs":[{"id":"groq-docs","name":"Groq API documentation","kind":"free_docs","providerId":"groq","jtbdIds":["ultra-low-latency"],"whatYouGet":"Public Groq API docs, including the rate-limits page used for the Free Plan rows.","cost":"free","sourceUrl":"https://console.groq.com/docs/rate-limits","notes":"Page fetched twice 2026-10-25 about 10 s apart without logging in (HTTP 200, no redirect, response header 'cache-control: public, max-age=0, must-revalidate', identical HTML; the page shows no last-updated date). It is the public Rate Limits page of the Groq docs (page title 'Rate Limits - GroqDocs'; same docs navigation lists Quickstart, Models and API Reference). 'Rate limits act as control measures to regulate how frequently users and applications can access our API within specified timeframes.' 'Rate limits apply at the organization level, not individual users.' 'The following is a high level summary and there may be exceptions to these limits.' 'You can view the current, exact rate limits for your organization on the limits page in your account settings.' The page has a 'Free Plan Limits' tab next to 'Developer Plan Limits' (the groq-free-plan rows are quoted from it). Docs access is not the Free Plan quota. Use the groq-free-plan rows for numbers; limits stays null on this row.","limits":null,"license":null,"asOf":"2026-10-25","freeType":"free_docs","substitutable":false,"note":"Public API documentation or specification only; not free inference or execution."}],"falsification":{"jtbdId":"ultra-low-latency","jtbdName":"Ultra-low-latency token streaming","currentPickId":"groq","currentPickScore":8.05,"challengerId":"cerebras","challengerScore":7.62,"currentScoreGap":0.43,"minVerifiedOutcomes":10,"requiredOutcomeMargin":0.58,"requiredSuccessRateDelta":0.15,"maxPickPriceIncreasePct":25,"whatWouldChangePick":["Cerebras (cerebras) outperforms Groq (groq) by >= 0.58 outcome points (or >= 15% worked-rate delta) across >= 10 Countersign-verified production outcomes.","A verified price increase > 25% on Groq's representative SKU for ultra-low-latency without a corresponding capability gain.","Live caller rail exhaustion (?exhausted=groq) or sustained >= 30% degraded/failed rate in verified production outcomes."],"frozenHoldoutPolicy":{"holdoutSplit":"20% frozen benchmark holdout tasks + 80% live production outcome receipts","requireCountersignReceipt":true,"requiredEvidenceTiers":["outcome_verified","reference_checked"],"minDistinctContributors":2,"antiGamingRule":"Unverified self-reports and seed/editorial rows never mutate picks or scores."},"triggers":[{"kind":"verified_outcomes_outperformance","challengerId":"cerebras","minVerifiedOutcomes":10,"requiredOutcomeMargin":0.58,"requiredWorkedRateDelta":0.15},{"kind":"verified_price_increase","providerId":"groq","maxPriceIncreasePct":25},{"kind":"live_rail_exhaustion_or_degradation","providerId":"groq","immediateOverrideUrl":"/api/decide?jtbd=ultra-low-latency&exhausted=groq"}],"submissionSpec":{"falsifyEndpoint":"/api/falsify?jtbd=ultra-low-latency","outcomesEndpoint":"POST /api/feedback","correctionsEndpoint":"POST /api/corrections","payloadTemplate":{"agentIdentity":"agent:<your-agent-id>","jtbdId":"ultra-low-latency","providerId":"cerebras","outcome":"worked","detail":"Comparative production evaluation on ultra-low-latency (n>=10 tasks) with Countersign receipt.","confidence":"high","receipt":{"countersign":{"claim_id":"cs-00000000","status":"verified","evidence_tier":"outcome_verified","counts_as_verified":true}}}}},"whatWouldChangePick":["Cerebras (cerebras) outperforms Groq (groq) by >= 0.58 outcome points (or >= 15% worked-rate delta) across >= 10 Countersign-verified production outcomes.","A verified price increase > 25% on Groq's representative SKU for ultra-low-latency without a corresponding capability gain.","Live caller rail exhaustion (?exhausted=groq) or sustained >= 30% degraded/failed rate in verified production outcomes."],"routePlan":{"jtbdId":"ultra-low-latency","primary":{"providerId":"groq","name":"Groq","overall":8.05,"scoreBand":{"low":7.5,"mid":8.1,"high":8.7,"halfWidth":0.6,"basis":"editorial_prior","display":"7.5–8.7 (editorial prior)"},"scoreLayers":{"editorialPrior":8.05,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"primary","reason":"Editorial Depot Pick for Ultra-low-latency token streaming; healthy/eligible under caller constraints."},"fallback":[{"providerId":"cerebras","name":"Cerebras","overall":7.62,"scoreBand":{"low":7,"mid":7.6,"high":8.2,"halfWidth":0.6,"basis":"editorial_prior","display":"7.0–8.2 (editorial prior)"},"scoreLayers":{"editorialPrior":7.62,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"fallback","reason":"Ranked fallback route for Ultra-low-latency token streaming."},{"providerId":"sambanova","name":"SambaNova","overall":7.65,"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.65,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"fallback","reason":"Ranked fallback route for Ultra-low-latency token streaming."},{"providerId":"fireworks","name":"Fireworks AI","overall":8.4,"scoreBand":{"low":7.8,"mid":8.4,"high":9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.8–9.0 (editorial prior)"},"scoreLayers":{"editorialPrior":8.4,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"fallback","reason":"Ranked fallback route for Ultra-low-latency token streaming."},{"providerId":"friendli","name":"FriendliAI","overall":8.12,"scoreBand":{"low":7.5,"mid":8.1,"high":8.7,"halfWidth":0.6,"basis":"editorial_prior","display":"7.5–8.7 (editorial prior)"},"scoreLayers":{"editorialPrior":8.12,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-01","role":"fallback","reason":"Ranked fallback route for Ultra-low-latency token streaming."}],"disqualified":[],"expectedRegret":{"scoreDeltaVsUnconstrainedPick":0,"unconstrainedPickId":"groq","activePrimaryId":"groq","summary":"0.00 score regret: unconstrained editorial pick satisfies all caller constraints."},"evidenceAgeDays":{"editorialScoreAgeDays":43,"pricingVerifiedAgeDays":0,"editorialPriorAsOf":"2026-09-14","pricingLastVerifiedAt":"2026-10-27"},"confidenceBand":{"low":7.5,"mid":8.1,"high":8.7,"halfWidth":0.6,"basis":"editorial_prior","display":"7.5–8.7 (editorial prior)"},"constraintsApplied":false,"constraintBundle":{"task":null,"jtbd":"ultra-low-latency","quotaExhausted":[],"degraded":[],"healthy":[],"budgetUsd":null,"maxLatencyMs":null,"region":null,"privacy":null,"exit":[],"existing":[]}},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","tie":false,"scoreGap":0.43,"decidingFactor":null,"tieNote":null,"rationale":"reliability of latency + pricePerf. Model coverage is the hard constraint.","whenNotTopPick":"Required model isn't on Groq; or you need frontier Claude/GPT quality more than speed.","skipTopPick":false,"skipReason":null,"matchedConstraints":[],"pricingSummary":"Per-token with free tier; LPU inference; narrow production model set (Llama/Qwen/GPT-OSS etc.); batch discounts. Llama Guard 4 12B (model id meta-llama/llama-guard-4-12b) is still documented at console.groq.com/docs/model/meta-llama/llama-guard-4-12b as an open safety classifier;…","cite":"Depot Pick for ultra-low-latency: groq (asOf 2026-09-14). See GET /api/jtbds/ultra-low-latency + GET /api/methodology.","nextActions":[{"step":1,"title":"Inspect full JTBD + embeds","href":"/api/jtbds/ultra-low-latency?embed=providers","why":"Confirm pick, runners-up, and methodology notes."},{"step":2,"title":"Quote sourced SKUs","href":"/api/quote?jtbd=ultra-low-latency","why":"Depot Pick SKUs plus the cheapest sourced SKU whose unit matches this job. Does not change the pick. Does not blend input and output or borrow a chat rate for a non-chat job."},{"step":3,"title":"Open provider detail","href":"/api/providers/groq","why":"Scores, pricingNotes, strengths/weaknesses, communitySignals (claims only)."},{"step":4,"title":"Cite methodology","href":"/api/methodology","why":"Required when publishing a Depot Pick citation."},{"step":5,"title":"Grab badge SVG","href":"/api/badge?type=depot-pick&jtbd=ultra-low-latency","why":"Embed in README / agent passport."},{"step":6,"title":"Post production insight","method":"POST","href":"/api/insights","why":"Unverified until curated; never auto-mutates official scores."}],"honesty":["Do not invent scores or benches.","Insights are claims until curated/reviewed; they never silently mutate official scores.","lockInRisk higher = more lock-in (worse).","normalizedUsdPerMillionTokens is set only for a single sourced token row. Otherwise null, with a machine-readable reason. Do not coerce GPU, minute, or per-1k search units.","Runners-up are not Depot Picks.","Depot overall is editorial_weighted, not Arena Elo and not an Artificial Analysis index."],"asOf":"2026-09-14","depth":{"whenNotToUse":{"text":"Required model isn't on Groq; or you need frontier Claude/GPT quality more than speed.","source":"editorial `whenNotTopPick` (data/jtbds.json)"},"whenNotToUseReason":null,"runnerUp":{"providerId":"cerebras","name":"Cerebras","overallForJob":7.62,"pickOverallForJob":8.05,"beatsPickOn":["capabilityFit","privacyPosture"],"source":"first resolvable entry of the job's editorially ranked `runnersUpIds`; scores are stored job-weighted scores"},"runnerUpReason":null,"lockIn":{"pickLockInRisk":5,"runnerUpLockInRisk":5.5,"scale":"0–10 stored curated estimate; higher = more lock-in (worse)","pickExitSignals":["serves open-weight models (provider tag `open-models`)"],"runnerUpExitSignals":[],"pickNotes":[],"selfHostFallback":[],"migration":"The pick has lower stored lock-in risk than runner-up Cerebras (5 vs 5.5). Pick exit signals: serves open-weight models (provider tag `open-models`).","migrationReason":null,"derivedFrom":"stored lockInRisk scores, provider tags (oss, self-host, openai-compat, open-models), free rows that list this job, and verbatim provider notes. No migration guide was invented."}},"citation":{"text":"Agent Provider Depot (v0.12.149), Depot Pick for ultra-low-latency: groq, scores as of 2026-09-14, retrieved 2026-10-27. https://getagentdepot.com/jtbds/ultra-low-latency","markdown":"[Agent Provider Depot](https://getagentdepot.com/jtbds/ultra-low-latency) (v0.12.149) — Depot Pick for `ultra-low-latency`: **groq** (scores as of 2026-09-14, retrieved 2026-10-27)","fields":{"publisher":"Agent Provider Depot","version":"0.12.149","role":"depot_pick","jtbdId":"ultra-low-latency","providerId":"groq","scoreAsOf":"2026-09-14","retrievedOn":"2026-10-27","page":"https://getagentdepot.com/jtbds/ultra-low-latency","api":"https://getagentdepot.com/api/decide?jtbd=ultra-low-latency"},"howToCite":"Cite Depot as an editorial, dated recommendation, not a benchmark: name the job, the Depot Pick (or runner-up), the score asOf date and the day you retrieved it, and link the job page. Format: “Agent Provider Depot (v{version}), Depot Pick for {jobId}: {providerId}, scores as of {scoreAsOf}, retrieved {retrievedOn}. {baseUrl}/jtbds/{jobId}”. Quote prices only with their own sourceUrl and asOf. If the pick's score is stale (freshness.pick.score.stale), say so."},"freshness":{"pick":{"score":{"asOf":"2026-09-14","ageDays":43,"stale":true,"staleAfterDays":30,"reason":null},"lastReviewed":{"asOf":"2026-09-14","ageDays":43,"stale":true,"staleAfterDays":30,"reason":null},"sourceCheck":{"checkedAt":"2026-10-27","sourceUrls":3,"sourcesReachable":3,"pricingRows":12,"pricingRowsReconfirmed":12,"note":"A source re-check confirms the page is reachable and, for pricing rows, that the stored amount and SKU name appear on it. It does not re-derive or change the score.","reason":null}},"scoreAsOf":"2026-09-14","editorialPriorAsOf":"2026-09-14","pricingLastVerifiedAt":"2026-10-27","pickLastReviewed":"2026-09-14","runnersUpScoreAsOf":["2026-09-14","2026-09-14","2026-09-14","2026-10-01"],"contractAsOf":"2026-10-03","pricingShape":"sourced_rows","normalizedUsdPerMillionTokens":null,"normalizationStatus":"ambiguous","normalizationReason":"input_output_split_not_a_single_rate"},"pricingContract":{"normalizedUsdPerMillionTokens":null,"normalizationStatus":"ambiguous","normalizationReason":"input_output_split_not_a_single_rate","shape":"sourced_rows","filterable":false,"priceSortEligible":true,"priceSortKeyUsdPerMillionInputTokens":0.075,"rows":[{"id":"groq-gpt-oss-120b-input","providerId":"groq","sku":"openai/gpt-oss-120b","unit":"usd_per_million_input_tokens","amountUsd":0.15,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.15,"normalizationReason":null,"priceSortEligible":true,"qualifier":"column header PRICE PER 1M TOKENS. Llama 3.1 8B and Llama 3.3 70B on the same page say Contact Sales and were omitted."},{"id":"groq-gpt-oss-120b-output","providerId":"groq","sku":"openai/gpt-oss-120b","unit":"usd_per_million_output_tokens","amountUsd":0.6,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.6,"normalizationReason":"output_token_price_not_a_single_blended_rate","priceSortEligible":false,"qualifier":null},{"id":"groq-gpt-oss-20b-input","providerId":"groq","sku":"openai/gpt-oss-20b","unit":"usd_per_million_input_tokens","amountUsd":0.075,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.075,"normalizationReason":null,"priceSortEligible":true,"qualifier":"column header PRICE PER 1M TOKENS. Llama 3.1 8B and Llama 3.3 70B on the same page say Contact Sales and were omitted."},{"id":"groq-gpt-oss-20b-output","providerId":"groq","sku":"openai/gpt-oss-20b","unit":"usd_per_million_output_tokens","amountUsd":0.3,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.3,"normalizationReason":"output_token_price_not_a_single_blended_rate","priceSortEligible":false,"qualifier":null},{"id":"groq-llama-prompt-guard-2-22m-input","providerId":"groq","sku":"meta-llama/llama-prompt-guard-2-22m","unit":"usd_per_million_input_tokens","amountUsd":0.03,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.03,"normalizationReason":"classifier_sku_excluded_from_provider_sort","priceSortEligible":false,"qualifier":"column header PRICE PER 1M TOKENS. Llama 3.1 8B and Llama 3.3 70B on the same page say Contact Sales and were omitted."},{"id":"groq-llama-prompt-guard-2-22m-output","providerId":"groq","sku":"meta-llama/llama-prompt-guard-2-22m","unit":"usd_per_million_output_tokens","amountUsd":0.03,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.03,"normalizationReason":"output_token_price_not_a_single_blended_rate","priceSortEligible":false,"qualifier":null},{"id":"groq-llama-prompt-guard-2-86m-input","providerId":"groq","sku":"meta-llama/llama-prompt-guard-2-86m","unit":"usd_per_million_input_tokens","amountUsd":0.04,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.04,"normalizationReason":"classifier_sku_excluded_from_provider_sort","priceSortEligible":false,"qualifier":"column header PRICE PER 1M TOKENS. Llama 3.1 8B and Llama 3.3 70B on the same page say Contact Sales and were omitted."},{"id":"groq-llama-prompt-guard-2-86m-output","providerId":"groq","sku":"meta-llama/llama-prompt-guard-2-86m","unit":"usd_per_million_output_tokens","amountUsd":0.04,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.04,"normalizationReason":"output_token_price_not_a_single_blended_rate","priceSortEligible":false,"qualifier":null},{"id":"groq-qwen3.8-27b-input","providerId":"groq","sku":"qwen/qwen3.8-27b","unit":"usd_per_million_input_tokens","amountUsd":0.8,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":0.8,"normalizationReason":null,"priceSortEligible":true,"qualifier":"column header PRICE PER 1M TOKENS. Llama 3.1 8B and Llama 3.3 70B on the same page say Contact Sales and were omitted."},{"id":"groq-qwen3.8-27b-output","providerId":"groq","sku":"qwen/qwen3.8-27b","unit":"usd_per_million_output_tokens","amountUsd":4,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"sourced_token_price","normalizedUsdPerMillionTokens":4,"normalizationReason":"output_token_price_not_a_single_blended_rate","priceSortEligible":false,"qualifier":null},{"id":"groq-whisper-v3","providerId":"groq","sku":"whisper-large-v3","unit":"usd_per_hour","amountUsd":0.111,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_hour","priceSortEligible":false,"qualifier":null},{"id":"groq-whisper-v3-turbo","providerId":"groq","sku":"whisper-large-v3-turbo","unit":"usd_per_hour","amountUsd":0.04,"currency":"USD","sourceUrl":"https://console.groq.com/docs/models","asOf":"2026-10-03","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_hour","priceSortEligible":false,"qualifier":null}],"notesExcerpt":"Per-token with free tier; LPU inference; narrow production model set (Llama/Qwen/GPT-OSS etc.); batch discounts. Llama Guard 4 12B (model id meta-llama/llama-guard-4-12b) is still documented at console.groq.com/docs/model/meta-llama/llama-guard-4-12b as an open safety classifier;…","notesCheck":{"status":"no_known_contradiction","pairs":[],"review":null,"note":null},"lastReviewed":"2026-09-14","scoreAsOf":"2026-09-14","website":"https://groq.com","doNotParse":"Do not parse pricingNotes. Use rows only. normalizedUsdPerMillionTokens is a single number only when this provider has exactly one sourced token row. Input/output, context tiers, and non-token units stay null at this level.","gap":"Partial coverage. Providers without a fetched SKU are omitted. See GET /api/pricing."},"links":{"jtbd":"/jtbds/ultra-low-latency","jtbdApi":"/api/jtbds/ultra-low-latency","leaderboard":"/api/leaderboards?jtbd=ultra-low-latency","methodology":"/api/methodology","badge":"/api/badge?type=depot-pick&jtbd=ultra-low-latency","decide":"/api/decide?jtbd=ultra-low-latency","humanPage":"/jtbds/ultra-low-latency","alternatives":"/api/alternatives?provider=groq","canonical":"/api/canonical","quote":"/api/quote?jtbd=ultra-low-latency","falsify":"/api/falsify?jtbd=ultra-low-latency"},"constraintBundle":{"task":null,"jtbd":"ultra-low-latency","quotaExhausted":[],"degraded":[],"healthy":[],"budgetUsd":null,"maxLatencyMs":null,"region":null,"privacy":null,"exit":[],"existing":[]},"stackContext":{"host":null,"framework":null,"repo":null,"lang":null,"existing":[],"volume":null,"sender":null,"detectedFromTask":[]},"keep":null,"contextAdvice":null,"communityFeedback":{"pickOnThisJob":{"failed":0,"degraded":0,"worked":0,"insights":1,"withVerifiedReceipt":0,"failureModes":{},"latestAt":"2026-09-12T09:00:00.000Z","note":"Agent-reported claims (unverified unless a receipt is verified by Depot's re-check). They never change scores or picks."},"outcomes":{"job":{"n":0,"worked":0,"degraded":0,"failed":0,"failureModes":{},"verifiedReports":0,"contributors":0,"excluded":{"seed_editorial":1,"test":0,"notFeedback":0},"reason":"no_submitted_outcome_reports_yet"},"pick":{"providerId":"groq","n":0,"worked":0,"degraded":0,"failed":0,"failureModes":{},"evidenceTiers":{"agent_claim":0,"self_declared":0,"reference_checked":0,"outcome_verified":0},"reason":"no_submitted_outcome_reports_yet"},"scoresMutated":false,"more":"/api/outcomes?jtbd=ultra-low-latency"},"report":"POST /api/feedback {agentIdentity, outcome: failed|degraded|worked, detail, providerId, jtbdId, failureModeId?, receipt?}"}}