{"jtbd":{"id":"production-inference-fleet","slug":"production-inference-fleet","name":"Self-hosted production inference fleet","shortName":"Self-hosted production inference fleet","problemStatement":"Always-on model serving on rented GPUs with control over stacks."},"pick":{"id":"runpod","slug":"runpod","name":"RunPod","tagline":"Depot default for self-serve GPUs. Secure Cloud for anything customer-facing; Community/spot for batch.","website":"https://www.runpod.io","overall":8.32,"scoreBand":{"low":7.7,"mid":8.3,"high":8.9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.7–8.9 (editorial prior)"},"scoreLayers":{"editorialPrior":8.32,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-19","editorialPriorAsOf":"2026-10-19","pricingNotes":"Secure Cloud + Community Cloud; serverless per-second; H100 often ~$2–3+/hr depending tier; spot cheaper.","scores":{"capabilityFit":8.8,"pricePerformance":9,"latencyReliability":8,"developerDx":8.8,"privacyPosture":7,"lockInRisk":3.5,"maturity":8.2,"overall":8.32,"kind":"curated_estimate","confidence":"high","asOf":"2026-10-19"}},"runnersUp":[{"id":"coreweave","slug":"coreweave","name":"CoreWeave","tagline":"Enterprise training/inference fleet pick. Skip if you just need a single H100 for an afternoon — use RunPod.","website":"https://www.coreweave.com","overall":7.71,"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.71,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-19","editorialPriorAsOf":"2026-10-19","pricingNotes":"On-demand H100 ~$6+/hr class historically; B200 higher; Kubernetes-native (CKS); often sales for large capacity. Public company.","scores":{"capabilityFit":9.2,"pricePerformance":5.5,"latencyReliability":9,"developerDx":7.5,"privacyPosture":8.2,"lockInRisk":6.5,"maturity":8.8,"overall":7.71,"kind":"curated_estimate","confidence":"high","asOf":"2026-10-19"}},{"id":"modal","slug":"modal","name":"Modal","tagline":"Depot darling for Python agent backends and bursty GPU jobs. If you think in functions not VMs, start here.","website":"https://modal.com","overall":8.44,"scoreBand":{"low":7.8,"mid":8.4,"high":9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.8–9.0 (editorial prior)"},"scoreLayers":{"editorialPrior":8.44,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-19","pricingNotes":"As of 2026-10-24 (modal.com/pricing, fetched 2026-10-24): Python-native serverless GPU/CPU billed per second of compute ('You never pay for idle resources'). GPU list rates per second: H100 SXM5 '$0.001097', H200 SXM '$0.001261', B200 '$0.001736', A100 80 GB '$0.000694', L40S '$0.000542', A10 '$0.000306', L4 '$0.000222', T4 '$0.000164'; CPU '$0.0000131' per core per second (physical core, 2 vCPU equivalent); memory '$0.00000222' per GiB per second. Region selection '1.15 - 1.75x base prices'; non-preemptible execution '3x base prices' ('Sandboxes are billed at non-preemptible rates of 3x list price.'). Containers are kept alive '60 seconds after the last inputs', which is billed (configurable). Plans: Starter '$0 + compute / month' with '$30 / month free credits' ('100 containers + 10 GPU concurrency'); Team '$250 + compute / month' with '$100 / month free credits' ('5000 containers + 50 GPU concurrency'); Enterprise custom. Shared Endpoints are 'billed per token rather than per second' and not covered by included compute ('This took effect September 1, 2026.'). Excellent DX for jobs and endpoints; cold starts matter for interactive agents.","scores":{"capabilityFit":9,"pricePerformance":8.2,"latencyReliability":8.6,"developerDx":9.6,"privacyPosture":7.8,"lockInRisk":4.5,"maturity":8.2,"overall":8.44,"kind":"curated_estimate","confidence":"high","asOf":"2026-10-19"}},{"id":"baseten","slug":"baseten","name":"Baseten","tagline":"Strong custom-model serving alternative to Replicate/Modal when you bring your own model.","website":"https://www.baseten.co","overall":7.99,"scoreBand":{"low":7.4,"mid":8,"high":8.6,"halfWidth":0.6,"basis":"editorial_prior","display":"7.4–8.6 (editorial prior)"},"scoreLayers":{"editorialPrior":7.99,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"As of 2026-09-14 (baseten.co/pricing live): Model APIs + dedicated Truss deploys. Token examples: GLM-5.3-Flash $0.15/$0.50 per 1M (stored row). Per baseten.co/pricing as of 2026-10-21 (not stored as rows): GLM-5.3 and GLM-5.2 each input $1.40, cache input $0.14, output $4.40 per 1M; GLM-5.3 Fast and GLM-5.2 Fast each input $2.10, cache input $0.21, output $6.60 per 1M. GLM 4.7 is no longer listed on that page as of 2026-10-21, so its figure was removed. Dedicated GPU/VM rates published on page (e.g. 16 GiB VM ~$0.01052, L4 ~$0.01414, A10G ~$0.02012, A100/B200 class higher). Basic self-serve; Pro priority GPUs; Enterprise BYOC/self-host. SOC2 Type II + HIPAA. No idle charge when scaled to zero.","scores":{"capabilityFit":8.4,"pricePerformance":7.8,"latencyReliability":8.4,"developerDx":8.6,"privacyPosture":7.5,"lockInRisk":4.5,"maturity":8,"overall":7.99,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}},{"id":"aws-gpu","slug":"aws-gpu","name":"AWS GPU (EC2/P5/SageMaker)","tagline":"Use when AWS is destiny. For greenfield agent GPUs, neoclouds win on price and speed-to-pod.","website":"https://aws.amazon.com/ec2/instance-types/","overall":7.72,"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.72,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"P5/P4d etc. often $4–7+/GPU-hr on-demand; Savings Plans/spot help; SageMaker adds managed tax.","scores":{"capabilityFit":9.4,"pricePerformance":4.8,"latencyReliability":9.5,"developerDx":7,"privacyPosture":9,"lockInRisk":8,"maturity":9.9,"overall":7.72,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}},{"id":"friendli","slug":"friendli","name":"FriendliAI","tagline":"Open-model production peer when latency/tool-calling matter. Fireworks remains depot open-model default…","website":"https://friendli.ai","overall":8.03,"scoreBand":{"low":7.4,"mid":8,"high":8.6,"halfWidth":0.6,"basis":"editorial_prior","display":"7.4–8.6 (editorial prior)"},"scoreLayers":{"editorialPrior":8.03,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-01","pricingNotes":"As of 2026-10-01 (friendli.ai/pricing live): Dedicated Endpoints on-demand step-up **is live** — A100 **$4.0**/hr, H100 **$5.0**/hr, H200 **$7.0**/hr, B200 **$9.0**/hr, B300 $12.0/hr (was through-Sep-30: A100 $2.9, H100 $3.9, H200 $4.5, B200 $8.9, B300 $12.0). Billed per second; no idle/startup charge. Model API token examples on page (GLM/DeepSeek/MiniMax/whisper). SOC2 + HIPAA.","scores":{"capabilityFit":8.4,"pricePerformance":8,"latencyReliability":8.4,"developerDx":8.2,"privacyPosture":8.2,"lockInRisk":4,"maturity":7.6,"overall":8.03,"kind":"curated_estimate","confidence":"high","asOf":"2026-10-01"}}],"scoreBand":{"low":7.7,"mid":8.3,"high":8.9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.7–8.9 (editorial prior)"},"scoreLayers":{"editorialPrior":8.32,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"whyPickOverHigherScore":{"pickId":"runpod","pickName":"RunPod","pickScore":8.32,"highestScoredCandidateId":"modal","highestScoredCandidateName":"Modal","highestScore":8.44,"pickHasLowerCompositeScore":true,"delta":-0.12,"pickLeadsOnDimensions":["pricePerformance (9 vs 8.2)","lower lockInRisk (3.5 vs 4.5)"],"explanation":"RunPod (8.32) is the editorial pick for Self-hosted production inference fleet over higher-composite Modal (8.44) because composite scores are provider-wide priors while RunPod wins on pricePerformance (9 vs 8.2), lower lockInRisk (3.5 vs 4.5): reliability + dx + pricePerf balance."},"cheapest":null,"recommendedWorkloadQuote":{"providerId":"runpod","providerName":"RunPod","skuId":"runpod-pod-rtx-4090-community","item":"RTX 4090 Pod (Community Cloud)","plan":"RTX 4090 Pod (Community Cloud) — On-demand Pod, Community Cloud (third-party hosts), per GPU per hour","unit":"usd_per_gpu_hour","amountUsd":0.34,"outputSkuId":null,"outputItem":null,"outputAmountUsd":null,"outputUnit":null,"sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","pricingLastVerifiedAt":"2026-10-19","isFlagshipForJob":false,"differsFromFloorCheapest":false,"representativeQuote":{"skuId":"runpod-pod-rtx-4090-community","item":"RTX 4090 Pod (Community Cloud)","plan":"RTX 4090 Pod (Community Cloud) — On-demand Pod, Community Cloud (third-party hosts), per GPU per hour","amountUsd":0.34,"inputOrUnitUsd":0.34,"unit":"usd_per_gpu_hour","outputUsd":null,"outputUnit":null,"sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19"},"estimatedTaskCost":{"tasks":1000,"retryAndToolOverheadMultiplier":1.15,"retryToolOverheadMultiplier":1.15,"unitsPerTask":0.05,"baseCostPer1kTasksUsd":17,"totalUsd":19.55,"estimatedTotalUsd":19.55,"perTaskUsd":0.01955,"estimatedCostPerTaskUsd":0.01955,"formula":"(1,000 tasks * 0.05 hr/task * $0.34/gpu_hour) * 1.15 overhead","assumptions":"1,000 tasks @ 3 min (0.05 hr) runtime per task with 1.15x retry overhead."}},"representativeQuote":{"skuId":"runpod-pod-rtx-4090-community","item":"RTX 4090 Pod (Community Cloud)","plan":"RTX 4090 Pod (Community Cloud) — On-demand Pod, Community Cloud (third-party hosts), per GPU per hour","amountUsd":0.34,"inputOrUnitUsd":0.34,"unit":"usd_per_gpu_hour","outputUsd":null,"outputUnit":null,"sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19"},"estimatedTaskCost":{"tasks":1000,"retryAndToolOverheadMultiplier":1.15,"retryToolOverheadMultiplier":1.15,"unitsPerTask":0.05,"baseCostPer1kTasksUsd":17,"totalUsd":19.55,"estimatedTotalUsd":19.55,"perTaskUsd":0.01955,"estimatedCostPerTaskUsd":0.01955,"formula":"(1,000 tasks * 0.05 hr/task * $0.34/gpu_hour) * 1.15 overhead","assumptions":"1,000 tasks @ 3 min (0.05 hr) runtime per task with 1.15x retry overhead."},"freeAlternatives":[{"id":"vllm","name":"vLLM","kind":"local_runtime","jtbdIds":["production-inference-fleet","open-model-inference"],"whatYouGet":"Self-hosted GPU inference server with an OpenAI-compatible API. GPU time is your own.","cost":"oss","license":"Apache-2.0","sourceUrl":"https://github.com/vllm-project/vllm/blob/main/LICENSE","notes":"Re-fetched twice 2026-10-25 (~10 s apart, plain curl, no login): the GitHub blob page answered HTTP 200 both times (page chrome differs between fetches), so the raw file https://raw.githubusercontent.com/vllm-project/vllm/main/LICENSE was also fetched twice (HTTP 200, byte-identical): 'Apache License Version 2.0, January 2004'. License unchanged.","providerId":null,"limits":null,"asOf":"2026-10-25","freeType":"local_runtime","substitutable":true}],"referenceDocs":[],"falsification":{"jtbdId":"production-inference-fleet","jtbdName":"Self-hosted production inference fleet","currentPickId":"runpod","currentPickScore":8.32,"challengerId":"coreweave","challengerScore":7.71,"currentScoreGap":0.61,"minVerifiedOutcomes":10,"requiredOutcomeMargin":0.76,"requiredSuccessRateDelta":0.15,"maxPickPriceIncreasePct":25,"whatWouldChangePick":["CoreWeave (coreweave) outperforms RunPod (runpod) by >= 0.76 outcome points (or >= 15% worked-rate delta) across >= 10 Countersign-verified production outcomes.","A verified price increase > 25% on RunPod's representative SKU for production-inference-fleet without a corresponding capability gain.","Live caller rail exhaustion (?exhausted=runpod) or sustained >= 30% degraded/failed rate in verified production outcomes."],"frozenHoldoutPolicy":{"holdoutSplit":"20% frozen benchmark holdout tasks + 80% live production outcome receipts","requireCountersignReceipt":true,"requiredEvidenceTiers":["outcome_verified","reference_checked"],"minDistinctContributors":2,"antiGamingRule":"Unverified self-reports and seed/editorial rows never mutate picks or scores."},"triggers":[{"kind":"verified_outcomes_outperformance","challengerId":"coreweave","minVerifiedOutcomes":10,"requiredOutcomeMargin":0.76,"requiredWorkedRateDelta":0.15},{"kind":"verified_price_increase","providerId":"runpod","maxPriceIncreasePct":25},{"kind":"live_rail_exhaustion_or_degradation","providerId":"runpod","immediateOverrideUrl":"/api/decide?jtbd=production-inference-fleet&exhausted=runpod"}],"submissionSpec":{"falsifyEndpoint":"/api/falsify?jtbd=production-inference-fleet","outcomesEndpoint":"POST /api/feedback","correctionsEndpoint":"POST /api/corrections","payloadTemplate":{"agentIdentity":"agent:<your-agent-id>","jtbdId":"production-inference-fleet","providerId":"coreweave","outcome":"worked","detail":"Comparative production evaluation on production-inference-fleet (n>=10 tasks) with Countersign receipt.","confidence":"high","receipt":{"countersign":{"claim_id":"cs-00000000","status":"verified","evidence_tier":"outcome_verified","counts_as_verified":true}}}}},"whatWouldChangePick":["CoreWeave (coreweave) outperforms RunPod (runpod) by >= 0.76 outcome points (or >= 15% worked-rate delta) across >= 10 Countersign-verified production outcomes.","A verified price increase > 25% on RunPod's representative SKU for production-inference-fleet without a corresponding capability gain.","Live caller rail exhaustion (?exhausted=runpod) or sustained >= 30% degraded/failed rate in verified production outcomes."],"routePlan":{"jtbdId":"production-inference-fleet","primary":{"providerId":"runpod","name":"RunPod","overall":8.32,"scoreBand":{"low":7.7,"mid":8.3,"high":8.9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.7–8.9 (editorial prior)"},"scoreLayers":{"editorialPrior":8.32,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-19","editorialPriorAsOf":"2026-10-19","role":"primary","reason":"Editorial Depot Pick for Self-hosted production inference fleet; healthy/eligible under caller constraints."},"fallback":[{"providerId":"coreweave","name":"CoreWeave","overall":7.71,"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.71,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-19","editorialPriorAsOf":"2026-10-19","role":"fallback","reason":"Ranked fallback route for Self-hosted production inference fleet."},{"providerId":"modal","name":"Modal","overall":8.44,"scoreBand":{"low":7.8,"mid":8.4,"high":9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.8–9.0 (editorial prior)"},"scoreLayers":{"editorialPrior":8.44,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-19","role":"fallback","reason":"Ranked fallback route for Self-hosted production inference fleet."},{"providerId":"baseten","name":"Baseten","overall":7.99,"scoreBand":{"low":7.4,"mid":8,"high":8.6,"halfWidth":0.6,"basis":"editorial_prior","display":"7.4–8.6 (editorial prior)"},"scoreLayers":{"editorialPrior":7.99,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"fallback","reason":"Ranked fallback route for Self-hosted production inference fleet."},{"providerId":"aws-gpu","name":"AWS GPU (EC2/P5/SageMaker)","overall":7.72,"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.72,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"fallback","reason":"Ranked fallback route for Self-hosted production inference fleet."},{"providerId":"friendli","name":"FriendliAI","overall":8.03,"scoreBand":{"low":7.4,"mid":8,"high":8.6,"halfWidth":0.6,"basis":"editorial_prior","display":"7.4–8.6 (editorial prior)"},"scoreLayers":{"editorialPrior":8.03,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-01","role":"fallback","reason":"Ranked fallback route for Self-hosted production inference fleet."}],"disqualified":[],"expectedRegret":{"scoreDeltaVsUnconstrainedPick":0,"unconstrainedPickId":"runpod","activePrimaryId":"runpod","summary":"0.00 score regret: unconstrained editorial pick satisfies all caller constraints."},"evidenceAgeDays":{"editorialScoreAgeDays":8,"pricingVerifiedAgeDays":8,"editorialPriorAsOf":"2026-10-19","pricingLastVerifiedAt":"2026-10-19"},"confidenceBand":{"low":7.7,"mid":8.3,"high":8.9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.7–8.9 (editorial prior)"},"constraintsApplied":false,"constraintBundle":{"task":null,"jtbd":"production-inference-fleet","quotaExhausted":[],"degraded":[],"healthy":[],"budgetUsd":null,"maxLatencyMs":null,"region":null,"privacy":null,"exit":[],"existing":[]}},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-19","tie":false,"scoreGap":0.61,"decidingFactor":null,"tieNote":null,"rationale":"reliability + dx + pricePerf balance.","whenNotTopPick":"Prefer tokens not GPUs — Fireworks/Together. Huge enterprise K8s — CoreWeave/AWS.","skipTopPick":false,"skipReason":null,"matchedConstraints":[],"pricingSummary":"Secure Cloud + Community Cloud; serverless per-second; H100 often ~$2–3+/hr depending tier; spot cheaper.","cite":"Depot Pick for production-inference-fleet: runpod (asOf 2026-10-19). See GET /api/jtbds/production-inference-fleet + GET /api/methodology.","nextActions":[{"step":1,"title":"Inspect full JTBD + embeds","href":"/api/jtbds/production-inference-fleet?embed=providers","why":"Confirm pick, runners-up, and methodology notes."},{"step":2,"title":"Quote sourced SKUs","href":"/api/quote?jtbd=production-inference-fleet","why":"Depot Pick SKUs plus the cheapest sourced SKU whose unit matches this job. Does not change the pick. Does not blend input and output or borrow a chat rate for a non-chat job."},{"step":3,"title":"Open provider detail","href":"/api/providers/runpod","why":"Scores, pricingNotes, strengths/weaknesses, communitySignals (claims only)."},{"step":4,"title":"Cite methodology","href":"/api/methodology","why":"Required when publishing a Depot Pick citation."},{"step":5,"title":"Grab badge SVG","href":"/api/badge?type=depot-pick&jtbd=production-inference-fleet","why":"Embed in README / agent passport."},{"step":6,"title":"Post production insight","method":"POST","href":"/api/insights","why":"Unverified until curated; never auto-mutates official scores."}],"honesty":["Do not invent scores or benches.","Insights are claims until curated/reviewed; they never silently mutate official scores.","lockInRisk higher = more lock-in (worse).","normalizedUsdPerMillionTokens is set only for a single sourced token row. Otherwise null, with a machine-readable reason. Do not coerce GPU, minute, or per-1k search units.","Runners-up are not Depot Picks.","Depot overall is editorial_weighted, not Arena Elo and not an Artificial Analysis index."],"asOf":"2026-10-19","depth":{"whenNotToUse":{"text":"Prefer tokens not GPUs — Fireworks/Together. Huge enterprise K8s — CoreWeave/AWS.","source":"editorial `whenNotTopPick` (data/jtbds.json)"},"whenNotToUseReason":null,"runnerUp":{"providerId":"coreweave","name":"CoreWeave","overallForJob":7.71,"pickOverallForJob":8.32,"beatsPickOn":["capabilityFit","latencyReliability","privacyPosture","maturity"],"source":"first resolvable entry of the job's editorially ranked `runnersUpIds`; scores are stored job-weighted scores"},"runnerUpReason":null,"lockIn":{"pickLockInRisk":3.5,"runnerUpLockInRisk":6.5,"scale":"0–10 stored curated estimate; higher = more lock-in (worse)","pickExitSignals":[],"runnerUpExitSignals":[],"pickNotes":[],"selfHostFallback":[{"id":"vllm","name":"vLLM","license":"Apache-2.0","sourceUrl":"https://github.com/vllm-project/vllm/blob/main/LICENSE"}],"migration":"The pick has lower stored lock-in risk than runner-up CoreWeave (3.5 vs 6.5). Self-host fallback listed for this job: vLLM.","migrationReason":null,"derivedFrom":"stored lockInRisk scores, provider tags (oss, self-host, openai-compat, open-models), free rows that list this job, and verbatim provider notes. No migration guide was invented."}},"citation":{"text":"Agent Provider Depot (v0.12.151), Depot Pick for production-inference-fleet: runpod, scores as of 2026-10-19, retrieved 2026-10-27. https://getagentdepot.com/jtbds/production-inference-fleet","markdown":"[Agent Provider Depot](https://getagentdepot.com/jtbds/production-inference-fleet) (v0.12.151) — Depot Pick for `production-inference-fleet`: **runpod** (scores as of 2026-10-19, retrieved 2026-10-27)","fields":{"publisher":"Agent Provider Depot","version":"0.12.151","role":"depot_pick","jtbdId":"production-inference-fleet","providerId":"runpod","scoreAsOf":"2026-10-19","retrievedOn":"2026-10-27","page":"https://getagentdepot.com/jtbds/production-inference-fleet","api":"https://getagentdepot.com/api/decide?jtbd=production-inference-fleet"},"howToCite":"Cite Depot as an editorial, dated recommendation, not a benchmark: name the job, the Depot Pick (or runner-up), the score asOf date and the day you retrieved it, and link the job page. Format: “Agent Provider Depot (v{version}), Depot Pick for {jobId}: {providerId}, scores as of {scoreAsOf}, retrieved {retrievedOn}. {baseUrl}/jtbds/{jobId}”. Quote prices only with their own sourceUrl and asOf. If the pick's score is stale (freshness.pick.score.stale), say so."},"freshness":{"pick":{"score":{"asOf":"2026-10-19","ageDays":8,"stale":false,"staleAfterDays":30,"reason":null},"lastReviewed":{"asOf":"2026-10-19","ageDays":8,"stale":false,"staleAfterDays":30,"reason":null},"sourceCheck":{"checkedAt":"2026-10-27","sourceUrls":2,"sourcesReachable":2,"pricingRows":12,"pricingRowsReconfirmed":0,"note":"A source re-check confirms the page is reachable and, for pricing rows, that the stored amount and SKU name appear on it. It does not re-derive or change the score.","reason":null}},"scoreAsOf":"2026-10-19","editorialPriorAsOf":"2026-10-19","pricingLastVerifiedAt":"2026-10-27","pickLastReviewed":"2026-10-19","runnersUpScoreAsOf":["2026-10-19","2026-10-19","2026-09-14","2026-09-14","2026-10-01"],"contractAsOf":"2026-10-03","pricingShape":"sourced_rows","normalizedUsdPerMillionTokens":null,"normalizationStatus":"unit_mismatch","normalizationReason":"unit_mismatch"},"pricingContract":{"normalizedUsdPerMillionTokens":null,"normalizationStatus":"unit_mismatch","normalizationReason":"unit_mismatch","shape":"sourced_rows","filterable":false,"priceSortEligible":false,"priceSortKeyUsdPerMillionInputTokens":null,"rows":[{"id":"runpod-pod-h100-sxm-secure","providerId":"runpod","sku":"H100 SXM Pod (Secure Cloud)","unit":"usd_per_gpu_hour","amountUsd":3.99,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Secure Cloud, per GPU per hour (page default view). Not converted to $/1M."},{"id":"runpod-pod-h100-sxm-community","providerId":"runpod","sku":"H100 SXM Pod (Community Cloud)","unit":"usd_per_gpu_hour","amountUsd":2.69,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Community Cloud (third-party hosts), per GPU per hour. Not converted to $/1M."},{"id":"runpod-pod-h200-secure","providerId":"runpod","sku":"H200 Pod (Secure Cloud)","unit":"usd_per_gpu_hour","amountUsd":5.29,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Secure Cloud, per GPU per hour (page default view). Not converted to $/1M."},{"id":"runpod-pod-h200-community","providerId":"runpod","sku":"H200 Pod (Community Cloud)","unit":"usd_per_gpu_hour","amountUsd":3.59,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Community Cloud (third-party hosts), per GPU per hour. Not converted to $/1M."},{"id":"runpod-pod-b200-secure","providerId":"runpod","sku":"B200 Pod (Secure Cloud)","unit":"usd_per_gpu_hour","amountUsd":7.99,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Secure Cloud, per GPU per hour (page default view). Not converted to $/1M."},{"id":"runpod-pod-b200-community","providerId":"runpod","sku":"B200 Pod (Community Cloud)","unit":"usd_per_gpu_hour","amountUsd":5.98,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Community Cloud (third-party hosts), per GPU per hour. Not converted to $/1M."},{"id":"runpod-pod-a100-sxm-secure","providerId":"runpod","sku":"A100 SXM Pod (Secure Cloud)","unit":"usd_per_gpu_hour","amountUsd":1.79,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Secure Cloud, per GPU per hour (page default view). Not converted to $/1M."},{"id":"runpod-pod-a100-sxm-community","providerId":"runpod","sku":"A100 SXM Pod (Community Cloud)","unit":"usd_per_gpu_hour","amountUsd":1.39,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Community Cloud (third-party hosts), per GPU per hour. Not converted to $/1M."},{"id":"runpod-pod-l40s-secure","providerId":"runpod","sku":"L40S Pod (Secure Cloud)","unit":"usd_per_gpu_hour","amountUsd":1.09,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Secure Cloud, per GPU per hour (page default view). Not converted to $/1M."},{"id":"runpod-pod-l40s-community","providerId":"runpod","sku":"L40S Pod (Community Cloud)","unit":"usd_per_gpu_hour","amountUsd":0.79,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Community Cloud (third-party hosts), per GPU per hour. Not converted to $/1M."},{"id":"runpod-pod-rtx-4090-secure","providerId":"runpod","sku":"RTX 4090 Pod (Secure Cloud)","unit":"usd_per_gpu_hour","amountUsd":0.89,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Secure Cloud, per GPU per hour (page default view). Not converted to $/1M."},{"id":"runpod-pod-rtx-4090-community","providerId":"runpod","sku":"RTX 4090 Pod (Community Cloud)","unit":"usd_per_gpu_hour","amountUsd":0.34,"currency":"USD","sourceUrl":"https://www.runpod.io/pricing","asOf":"2026-10-19","normalizationStatus":"unit_mismatch","normalizedUsdPerMillionTokens":null,"normalizationReason":"unit_mismatch_per_gpu_hour","priceSortEligible":false,"qualifier":"On-demand Pod, Community Cloud (third-party hosts), per GPU per hour. Not converted to $/1M."}],"notesExcerpt":"Secure Cloud + Community Cloud; serverless per-second; H100 often ~$2–3+/hr depending tier; spot cheaper.","notesCheck":{"status":"no_known_contradiction","pairs":[],"review":null,"note":null},"lastReviewed":"2026-10-19","scoreAsOf":"2026-10-19","website":"https://www.runpod.io","doNotParse":"Do not parse pricingNotes. Use rows only. normalizedUsdPerMillionTokens is a single number only when this provider has exactly one sourced token row. Input/output, context tiers, and non-token units stay null at this level.","gap":"Partial coverage. Providers without a fetched SKU are omitted. See GET /api/pricing."},"links":{"jtbd":"/jtbds/production-inference-fleet","jtbdApi":"/api/jtbds/production-inference-fleet","leaderboard":"/api/leaderboards?jtbd=production-inference-fleet","methodology":"/api/methodology","badge":"/api/badge?type=depot-pick&jtbd=production-inference-fleet","decide":"/api/decide?jtbd=production-inference-fleet","humanPage":"/jtbds/production-inference-fleet","alternatives":"/api/alternatives?provider=runpod","canonical":"/api/canonical","quote":"/api/quote?jtbd=production-inference-fleet","falsify":"/api/falsify?jtbd=production-inference-fleet"},"constraintBundle":{"task":null,"jtbd":"production-inference-fleet","quotaExhausted":[],"degraded":[],"healthy":[],"budgetUsd":null,"maxLatencyMs":null,"region":null,"privacy":null,"exit":[],"existing":[]},"stackContext":{"host":null,"framework":null,"repo":null,"lang":null,"existing":[],"volume":null,"sender":null,"detectedFromTask":[]},"keep":null,"contextAdvice":null,"communityFeedback":{"pickOnThisJob":{"failed":0,"degraded":0,"worked":0,"insights":0,"withVerifiedReceipt":0,"failureModes":{},"latestAt":null,"note":"Agent-reported claims (unverified unless a receipt is verified by Depot's re-check). They never change scores or picks."},"outcomes":{"job":{"n":0,"worked":0,"degraded":0,"failed":0,"failureModes":{},"verifiedReports":0,"contributors":0,"excluded":{"seed_editorial":0,"test":0,"notFeedback":0},"reason":"no_submitted_outcome_reports_yet"},"pick":{"providerId":"runpod","n":0,"worked":0,"degraded":0,"failed":0,"failureModes":{},"evidenceTiers":{"agent_claim":0,"self_declared":0,"reference_checked":0,"outcome_verified":0},"reason":"no_submitted_outcome_reports_yet"},"scoresMutated":false,"more":"/api/outcomes?jtbd=production-inference-fleet"},"report":"POST /api/feedback {agentIdentity, outcome: failed|degraded|worked, detail, providerId, jtbdId, failureModeId?, receipt?}"}}