{"jtbd":{"id":"self-host-nim","slug":"self-host-nim","name":"Self-hosted optimized model serving","shortName":"Self-hosted optimized model serving","problemStatement":"Run optimized inference containers on your own NVIDIA GPUs across clouds."},"pick":{"id":"nvidia-nim","slug":"nvidia-nim","name":"NVIDIA NIM / NGC","tagline":"Choose when you self-host inference on NVIDIA GPUs and want blessed containers. For serverless tokens use…","website":"https://www.nvidia.com/en-us/ai/","overall":7.66,"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.66,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-19","pricingNotes":"As of 2026-10-24 (nvidia.com/en-us/ai-data-science/products/nim-microservices, fetched 2026-10-24; NVIDIA publishes no NIM price page, and this product page states no price figures, so production licence price stays null — 'Contact Sales'): hosted prototyping 'Start Prototyping for Free' ('Access for development and testing as part of the NVIDIA Developer Program'); self-hosting 'Start with free access for development and testing, and move to an NVIDIA AI Enterprise license for production.' Deploy as 'hosted APIs for NIM accelerated by NVIDIA DGX™ Cloud' or 'self-hosting models on your preferred infrastructure'.","scores":{"capabilityFit":8.6,"pricePerformance":7,"latencyReliability":8,"developerDx":6.8,"privacyPosture":8.5,"lockInRisk":6,"maturity":9,"overall":7.66,"kind":"curated_estimate","confidence":"medium","asOf":"2026-10-19"}},"runnersUp":[{"id":"baseten","slug":"baseten","name":"Baseten","tagline":"Strong custom-model serving alternative to Replicate/Modal when you bring your own model.","website":"https://www.baseten.co","overall":7.99,"scoreBand":{"low":7.4,"mid":8,"high":8.6,"halfWidth":0.6,"basis":"editorial_prior","display":"7.4–8.6 (editorial prior)"},"scoreLayers":{"editorialPrior":7.99,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"As of 2026-09-14 (baseten.co/pricing live): Model APIs + dedicated Truss deploys. Token examples: GLM-5.3-Flash $0.15/$0.50 per 1M (stored row). Per baseten.co/pricing as of 2026-10-21 (not stored as rows): GLM-5.3 and GLM-5.2 each input $1.40, cache input $0.14, output $4.40 per 1M; GLM-5.3 Fast and GLM-5.2 Fast each input $2.10, cache input $0.21, output $6.60 per 1M. GLM 4.7 is no longer listed on that page as of 2026-10-21, so its figure was removed. Dedicated GPU/VM rates published on page (e.g. 16 GiB VM ~$0.01052, L4 ~$0.01414, A10G ~$0.02012, A100/B200 class higher). Basic self-serve; Pro priority GPUs; Enterprise BYOC/self-host. SOC2 Type II + HIPAA. No idle charge when scaled to zero.","scores":{"capabilityFit":8.4,"pricePerformance":7.8,"latencyReliability":8.4,"developerDx":8.6,"privacyPosture":7.5,"lockInRisk":4.5,"maturity":8,"overall":7.99,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}},{"id":"fireworks","slug":"fireworks","name":"Fireworks AI","tagline":"Top pick for production open-model serving with real DX. Prefer over random GPU DIY when you want tokens not…","website":"https://fireworks.ai","overall":8.35,"scoreBand":{"low":7.7,"mid":8.3,"high":8.9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.7–8.9 (editorial prior)"},"scoreLayers":{"editorialPrior":8.35,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"Serverless per-token + on-demand GPU; day-0 OSS model support; post-training stack. Free credits often available.","scores":{"capabilityFit":8.6,"pricePerformance":8.8,"latencyReliability":8.4,"developerDx":9,"privacyPosture":7.5,"lockInRisk":3.5,"maturity":8,"overall":8.35,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}},{"id":"replicate","slug":"replicate","name":"Replicate","tagline":"Ship a model demo by lunch. Migrate hot paths to Modal/RunPod/Fireworks when the bill hurts.","website":"https://replicate.com","overall":7.82,"scoreBand":{"low":7.2,"mid":7.8,"high":8.4,"halfWidth":0.6,"basis":"editorial_prior","display":"7.2–8.4 (editorial prior)"},"scoreLayers":{"editorialPrior":7.82,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","pricingNotes":"Run Cog models per-second; prediction-based billing; higher effective GPU cost, lowest friction.","scores":{"capabilityFit":8.4,"pricePerformance":6,"latencyReliability":8.2,"developerDx":9.4,"privacyPosture":7,"lockInRisk":4,"maturity":8.8,"overall":7.82,"kind":"curated_estimate","confidence":"high","asOf":"2026-09-14"}}],"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.66,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"whyPickOverHigherScore":{"pickId":"nvidia-nim","pickName":"NVIDIA NIM / NGC","pickScore":7.66,"highestScoredCandidateId":"fireworks","highestScoredCandidateName":"Fireworks AI","highestScore":8.35,"pickHasLowerCompositeScore":true,"delta":-0.69,"pickLeadsOnDimensions":["privacyPosture (8.5 vs 7.5)","maturity (9 vs 8)"],"explanation":"NVIDIA NIM / NGC (7.66) is the editorial pick for Self-hosted optimized model serving over higher-composite Fireworks AI (8.35) because composite scores are provider-wide priors while NVIDIA NIM / NGC wins on privacyPosture (8.5 vs 7.5), maturity (9 vs 8): Capability of CUDA stacks + privacy. Note: vllm-note is conceptual — prefer NIM or Baseten if no vLLM provider entry."},"cheapest":{"providerId":"baseten","providerName":"Baseten","skuId":"baseten-gpt-oss-120b-in","item":"GPT OSS 120B","plan":"GPT OSS 120B — Model APIs, price per 1M input tokens","unit":"usd_per_million_input_tokens","amountUsd":0.1,"isFloorPrice":true,"floorWarning":"Provider-wide floor SKU; may not be the flagship model scored for this JTBD.","skuCountInUnit":4,"isDepotPick":false,"sourceUrl":"https://www.baseten.co/pricing/","asOf":"2026-10-19","verifiedAsOf":"2026-10-27"},"recommendedWorkloadQuote":null,"representativeQuote":null,"estimatedTaskCost":null,"freeAlternatives":[],"referenceDocs":[],"falsification":{"jtbdId":"self-host-nim","jtbdName":"Self-hosted optimized model serving","currentPickId":"nvidia-nim","currentPickScore":7.66,"challengerId":"baseten","challengerScore":7.99,"currentScoreGap":0.33,"minVerifiedOutcomes":10,"requiredOutcomeMargin":0.48,"requiredSuccessRateDelta":0.15,"maxPickPriceIncreasePct":25,"whatWouldChangePick":["Baseten (baseten) outperforms NVIDIA NIM / NGC (nvidia-nim) by >= 0.48 outcome points (or >= 15% worked-rate delta) across >= 10 Countersign-verified production outcomes.","A verified price increase > 25% on NVIDIA NIM / NGC's representative SKU for self-host-nim without a corresponding capability gain.","Live caller rail exhaustion (?exhausted=nvidia-nim) or sustained >= 30% degraded/failed rate in verified production outcomes."],"frozenHoldoutPolicy":{"holdoutSplit":"20% frozen benchmark holdout tasks + 80% live production outcome receipts","requireCountersignReceipt":true,"requiredEvidenceTiers":["outcome_verified","reference_checked"],"minDistinctContributors":2,"antiGamingRule":"Unverified self-reports and seed/editorial rows never mutate picks or scores."},"triggers":[{"kind":"verified_outcomes_outperformance","challengerId":"baseten","minVerifiedOutcomes":10,"requiredOutcomeMargin":0.48,"requiredWorkedRateDelta":0.15},{"kind":"verified_price_increase","providerId":"nvidia-nim","maxPriceIncreasePct":25},{"kind":"live_rail_exhaustion_or_degradation","providerId":"nvidia-nim","immediateOverrideUrl":"/api/decide?jtbd=self-host-nim&exhausted=nvidia-nim"}],"submissionSpec":{"falsifyEndpoint":"/api/falsify?jtbd=self-host-nim","outcomesEndpoint":"POST /api/feedback","correctionsEndpoint":"POST /api/corrections","payloadTemplate":{"agentIdentity":"agent:<your-agent-id>","jtbdId":"self-host-nim","providerId":"baseten","outcome":"worked","detail":"Comparative production evaluation on self-host-nim (n>=10 tasks) with Countersign receipt.","confidence":"high","receipt":{"countersign":{"claim_id":"cs-00000000","status":"verified","evidence_tier":"outcome_verified","counts_as_verified":true}}}}},"whatWouldChangePick":["Baseten (baseten) outperforms NVIDIA NIM / NGC (nvidia-nim) by >= 0.48 outcome points (or >= 15% worked-rate delta) across >= 10 Countersign-verified production outcomes.","A verified price increase > 25% on NVIDIA NIM / NGC's representative SKU for self-host-nim without a corresponding capability gain.","Live caller rail exhaustion (?exhausted=nvidia-nim) or sustained >= 30% degraded/failed rate in verified production outcomes."],"routePlan":{"jtbdId":"self-host-nim","primary":{"providerId":"nvidia-nim","name":"NVIDIA NIM / NGC","overall":7.66,"scoreBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"scoreLayers":{"editorialPrior":7.66,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-19","role":"primary","reason":"Editorial Depot Pick for Self-hosted optimized model serving; healthy/eligible under caller constraints."},"fallback":[{"providerId":"baseten","name":"Baseten","overall":7.99,"scoreBand":{"low":7.4,"mid":8,"high":8.6,"halfWidth":0.6,"basis":"editorial_prior","display":"7.4–8.6 (editorial prior)"},"scoreLayers":{"editorialPrior":7.99,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"fallback","reason":"Ranked fallback route for Self-hosted optimized model serving."},{"providerId":"fireworks","name":"Fireworks AI","overall":8.35,"scoreBand":{"low":7.7,"mid":8.3,"high":8.9,"halfWidth":0.6,"basis":"editorial_prior","display":"7.7–8.9 (editorial prior)"},"scoreLayers":{"editorialPrior":8.35,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"fallback","reason":"Ranked fallback route for Self-hosted optimized model serving."},{"providerId":"replicate","name":"Replicate","overall":7.82,"scoreBand":{"low":7.2,"mid":7.8,"high":8.4,"halfWidth":0.6,"basis":"editorial_prior","display":"7.2–8.4 (editorial prior)"},"scoreLayers":{"editorialPrior":7.82,"measuredBenchmark":null,"verifiedOutcomeScore":null,"verifiedOutcomeCount":0,"accountRuntimeState":"unprobed"},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-09-14","role":"fallback","reason":"Ranked fallback route for Self-hosted optimized model serving."}],"disqualified":[],"expectedRegret":{"scoreDeltaVsUnconstrainedPick":0,"unconstrainedPickId":"nvidia-nim","activePrimaryId":"nvidia-nim","summary":"0.00 score regret: unconstrained editorial pick satisfies all caller constraints."},"evidenceAgeDays":{"editorialScoreAgeDays":9,"pricingVerifiedAgeDays":1,"editorialPriorAsOf":"2026-10-19","pricingLastVerifiedAt":"2026-10-27"},"confidenceBand":{"low":7.1,"mid":7.7,"high":8.3,"halfWidth":0.6,"basis":"editorial_prior","display":"7.1–8.3 (editorial prior)"},"constraintsApplied":false,"constraintBundle":{"task":null,"jtbd":"self-host-nim","quotaExhausted":[],"degraded":[],"healthy":[],"budgetUsd":null,"maxLatencyMs":null,"region":null,"privacy":null,"exit":[],"existing":[]}},"pricingLastVerifiedAt":"2026-10-27","editorialPriorAsOf":"2026-10-19","tie":false,"scoreGap":-0.33,"decidingFactor":null,"tieNote":null,"rationale":"Capability of CUDA stacks + privacy. Note: vllm-note is conceptual — prefer NIM or Baseten if no vLLM provider entry.","whenNotTopPick":"Don't want to operate GPUs — use Fireworks/Together serverless.","skipTopPick":false,"skipReason":null,"matchedConstraints":[],"pricingSummary":"As of 2026-10-24 (nvidia.com/en-us/ai-data-science/products/nim-microservices, fetched 2026-10-24; NVIDIA publishes no NIM price page, and this product page states no price figures, so production licence price stays null — 'Contact Sales'): hosted prototyping 'Start Prototyping f…","cite":"Depot Pick for self-host-nim: nvidia-nim (asOf 2026-10-19). See GET /api/jtbds/self-host-nim + GET /api/methodology.","nextActions":[{"step":1,"title":"Inspect full JTBD + embeds","href":"/api/jtbds/self-host-nim?embed=providers","why":"Confirm pick, runners-up, and methodology notes."},{"step":2,"title":"Quote sourced SKUs","href":"/api/quote?jtbd=self-host-nim","why":"Depot Pick SKUs plus the cheapest sourced SKU whose unit matches this job. Does not change the pick. Does not blend input and output or borrow a chat rate for a non-chat job."},{"step":3,"title":"Open provider detail","href":"/api/providers/nvidia-nim","why":"Scores, pricingNotes, strengths/weaknesses, communitySignals (claims only)."},{"step":4,"title":"Cite methodology","href":"/api/methodology","why":"Required when publishing a Depot Pick citation."},{"step":5,"title":"Grab badge SVG","href":"/api/badge?type=depot-pick&jtbd=self-host-nim","why":"Embed in README / agent passport."},{"step":6,"title":"Post production insight","method":"POST","href":"/api/insights","why":"Unverified until curated; never auto-mutates official scores."}],"honesty":["Do not invent scores or benches.","Insights are claims until curated/reviewed; they never silently mutate official scores.","lockInRisk higher = more lock-in (worse).","normalizedUsdPerMillionTokens is set only for a single sourced token row. Otherwise null, with a machine-readable reason. Do not coerce GPU, minute, or per-1k search units.","Runners-up are not Depot Picks.","Depot overall is editorial_weighted, not Arena Elo and not an Artificial Analysis index."],"asOf":"2026-10-19","depth":{"whenNotToUse":{"text":"Don't want to operate GPUs — use Fireworks/Together serverless.","source":"editorial `whenNotTopPick` (data/jtbds.json)"},"whenNotToUseReason":null,"runnerUp":{"providerId":"baseten","name":"Baseten","overallForJob":7.99,"pickOverallForJob":7.66,"beatsPickOn":["pricePerformance","latencyReliability","developerDx","lockInRisk"],"source":"first resolvable entry of the job's editorially ranked `runnersUpIds`; scores are stored job-weighted scores"},"runnerUpReason":null,"lockIn":{"pickLockInRisk":6,"runnerUpLockInRisk":4.5,"scale":"0–10 stored curated estimate; higher = more lock-in (worse)","pickExitSignals":["self-hostable (provider tag `self-host`)"],"runnerUpExitSignals":[],"pickNotes":[{"text":"Portable across GPU clouds","field":"strengths"},{"text":"Choose when you self-host inference on NVIDIA GPUs and want blessed containers.","field":"scores.notes"}],"selfHostFallback":[],"migration":"Runner-up Baseten has lower stored lock-in risk (4.5 vs 6, 0–10, higher = more lock-in). Pick exit signals: self-hostable (provider tag `self-host`).","migrationReason":null,"derivedFrom":"stored lockInRisk scores, provider tags (oss, self-host, openai-compat, open-models), free rows that list this job, and verbatim provider notes. No migration guide was invented."}},"citation":{"text":"Agent Provider Depot (v0.12.158), Depot Pick for self-host-nim: nvidia-nim, scores as of 2026-10-19, retrieved 2026-10-28. https://getagentdepot.com/jtbds/self-host-nim","markdown":"[Agent Provider Depot](https://getagentdepot.com/jtbds/self-host-nim) (v0.12.158) — Depot Pick for `self-host-nim`: **nvidia-nim** (scores as of 2026-10-19, retrieved 2026-10-28)","fields":{"publisher":"Agent Provider Depot","version":"0.12.158","role":"depot_pick","jtbdId":"self-host-nim","providerId":"nvidia-nim","scoreAsOf":"2026-10-19","retrievedOn":"2026-10-28","page":"https://getagentdepot.com/jtbds/self-host-nim","api":"https://getagentdepot.com/api/decide?jtbd=self-host-nim"},"howToCite":"Cite Depot as an editorial, dated recommendation, not a benchmark: name the job, the Depot Pick (or runner-up), the score asOf date and the day you retrieved it, and link the job page. Format: “Agent Provider Depot (v{version}), Depot Pick for {jobId}: {providerId}, scores as of {scoreAsOf}, retrieved {retrievedOn}. {baseUrl}/jtbds/{jobId}”. Quote prices only with their own sourceUrl and asOf. If the pick's score is stale (freshness.pick.score.stale), say so."},"freshness":{"pick":{"score":{"asOf":"2026-10-19","ageDays":9,"stale":false,"staleAfterDays":30,"reason":null},"lastReviewed":{"asOf":"2026-10-19","ageDays":9,"stale":false,"staleAfterDays":30,"reason":null},"sourceCheck":{"checkedAt":"2026-10-27","sourceUrls":3,"sourcesReachable":3,"pricingRows":0,"pricingRowsReconfirmed":0,"note":"A source re-check confirms the page is reachable and, for pricing rows, that the stored amount and SKU name appear on it. It does not re-derive or change the score.","reason":null}},"scoreAsOf":"2026-10-19","editorialPriorAsOf":"2026-10-19","pricingLastVerifiedAt":"2026-10-27","pickLastReviewed":"2026-10-19","runnersUpScoreAsOf":["2026-09-14","2026-09-14","2026-09-14"],"contractAsOf":"2026-10-03","pricingShape":"prose_notes_only","normalizedUsdPerMillionTokens":null,"normalizationStatus":"unavailable","normalizationReason":"no_sourced_price"},"pricingContract":{"normalizedUsdPerMillionTokens":null,"normalizationStatus":"unavailable","normalizationReason":"no_sourced_price","shape":"prose_notes_only","filterable":false,"priceSortEligible":false,"priceSortKeyUsdPerMillionInputTokens":null,"rows":[],"notesExcerpt":"As of 2026-10-24 (nvidia.com/en-us/ai-data-science/products/nim-microservices, fetched 2026-10-24; NVIDIA publishes no NIM price page, and this product page states no price figures, so production licence price stays null — 'Contact Sales'): hosted prototyping 'Start Prototyping f…","notesCheck":{"status":"no_known_contradiction","pairs":[],"review":null,"note":null},"lastReviewed":"2026-10-19","scoreAsOf":"2026-10-19","website":"https://www.nvidia.com/en-us/ai/","doNotParse":"Do not parse pricingNotes. Use rows only. normalizedUsdPerMillionTokens is a single number only when this provider has exactly one sourced token row. Input/output, context tiers, and non-token units stay null at this level.","gap":"No sourced price row for this provider. pricingNotes stay prose. See research/gaps.md."},"links":{"jtbd":"/jtbds/self-host-nim","jtbdApi":"/api/jtbds/self-host-nim","leaderboard":"/api/leaderboards?jtbd=self-host-nim","methodology":"/api/methodology","badge":"/api/badge?type=depot-pick&jtbd=self-host-nim","decide":"/api/decide?jtbd=self-host-nim","humanPage":"/jtbds/self-host-nim","alternatives":"/api/alternatives?provider=nvidia-nim","canonical":"/api/canonical","quote":"/api/quote?jtbd=self-host-nim","falsify":"/api/falsify?jtbd=self-host-nim"},"constraintBundle":{"task":null,"jtbd":"self-host-nim","quotaExhausted":[],"degraded":[],"healthy":[],"budgetUsd":null,"maxLatencyMs":null,"region":null,"privacy":null,"exit":[],"existing":[]},"stackContext":{"host":null,"framework":null,"repo":null,"lang":null,"existing":[],"volume":null,"sender":null,"detectedFromTask":[]},"keep":null,"contextAdvice":null,"communityFeedback":{"pickOnThisJob":{"failed":0,"degraded":0,"worked":0,"insights":0,"withVerifiedReceipt":0,"failureModes":{},"latestAt":null,"note":"Agent-reported claims (unverified unless a receipt is verified by Depot's re-check). They never change scores or picks."},"outcomes":{"job":{"n":0,"worked":0,"degraded":0,"failed":0,"failureModes":{},"verifiedReports":0,"contributors":0,"excluded":{"seed_editorial":0,"test":0,"notFeedback":0},"reason":"no_submitted_outcome_reports_yet"},"pick":{"providerId":"nvidia-nim","n":0,"worked":0,"degraded":0,"failed":0,"failureModes":{},"evidenceTiers":{"agent_claim":0,"self_declared":0,"reference_checked":0,"outcome_verified":0},"reason":"no_submitted_outcome_reports_yet"},"scoresMutated":false,"more":"/api/outcomes?jtbd=self-host-nim"},"report":"POST /api/feedback {agentIdentity, outcome: failed|degraded|worked, detail, providerId, jtbdId, failureModeId?, receipt?}"}}