{"id":"ultra-low-latency","slug":"ultra-low-latency","name":"Ultra-low-latency token streaming","shortName":"Ultra-low-latency token streaming","problemStatement":"Voice/realtime UX needs extremely fast tokens/sec on supported models.","whenNotTopPick":"Required model isn't on Groq; or you need frontier Claude/GPT quality more than speed.","categoryIds":["cat-inference-apis"],"depotPickProviderId":"groq","depotPickRationale":"reliability of latency + pricePerf. Model coverage is the hard constraint.","runnersUpIds":["cerebras","sambanova","fireworks","friendli"],"weights":{"capabilityFit":0.2,"pricePerformance":0.15,"latencyReliability":0.3,"developerDx":0.15,"privacyPosture":0.08,"lockInRisk":0.05,"maturity":0.07},"methodologyNotes":"reliability of latency + pricePerf. Model coverage is the hard constraint.","tags":["ultra"],"depth":{"whenNotToUse":{"text":"Required model isn't on Groq; or you need frontier Claude/GPT quality more than speed.","source":"editorial `whenNotTopPick` (data/jtbds.json)"},"whenNotToUseReason":null,"runnerUp":{"providerId":"cerebras","name":"Cerebras","overallForJob":7.62,"pickOverallForJob":8.05,"beatsPickOn":["capabilityFit","privacyPosture"],"source":"first resolvable entry of the job's editorially ranked `runnersUpIds`; scores are stored job-weighted scores"},"runnerUpReason":null,"lockIn":{"pickLockInRisk":5,"runnerUpLockInRisk":5.5,"scale":"0–10 stored curated estimate; higher = more lock-in (worse)","pickExitSignals":["serves open-weight models (provider tag `open-models`)"],"runnerUpExitSignals":[],"pickNotes":[],"selfHostFallback":[],"migration":"The pick has lower stored lock-in risk than runner-up Cerebras (5 vs 5.5). Pick exit signals: serves open-weight models (provider tag `open-models`).","migrationReason":null,"derivedFrom":"stored lockInRisk scores, provider tags (oss, self-host, openai-compat, open-models), free rows that list this job, and verbatim provider notes. No migration guide was invented."}},"freshness":{"pick":{"score":{"asOf":"2026-09-14","ageDays":43,"stale":true,"staleAfterDays":30,"reason":null},"lastReviewed":{"asOf":"2026-09-14","ageDays":43,"stale":true,"staleAfterDays":30,"reason":null},"sourceCheck":{"checkedAt":"2026-10-27","sourceUrls":3,"sourcesReachable":3,"pricingRows":12,"pricingRowsReconfirmed":12,"note":"A source re-check confirms the page is reachable and, for pricing rows, that the stored amount and SKU name appear on it. It does not re-derive or change the score.","reason":null}}},"depotPick":{"id":"groq","slug":"groq","name":"Groq","tagline":"Use when latency is the product. Do not build your whole company on one Groq model SKU — they rotate.","overall":8.05},"runnersUp":[{"id":"cerebras","slug":"cerebras","name":"Cerebras","tagline":"Best ultra-throughput play when your model is on Cerebras (official token matrix now pinned). Pair with…","overall":7.62},{"id":"sambanova","slug":"sambanova","name":"SambaNova","tagline":"Public token matrix now exists — gpt-oss-120b $0.22/$0.59 is the value SKU. Indie agents still default…","overall":7.65},{"id":"fireworks","slug":"fireworks","name":"Fireworks AI","tagline":"Top pick for production open-model serving with real DX. Prefer over random GPU DIY when you want tokens not…","overall":8.4},{"id":"friendli","slug":"friendli","name":"FriendliAI","tagline":"Open-model production peer when latency/tool-calling matter. Fireworks remains depot open-model default…","overall":8.12}]}