Self-hosted optimized model serving
Run optimized inference containers on your own NVIDIA GPUs across clouds.
At a glance — Oct 2026
Updated 2026-10-27 · forward this page: /jtbds/self-host-nim#briefScores as of 2026-10-19 (8 days old).
- Depot Pick
- NVIDIA NIM / NGC · 7.66/10 for this job (curated estimate, medium confidence)
Capability of CUDA stacks + privacy. Note: vllm-note is conceptual — prefer NIM or Baseten if no vLLM provider entry.
- Runner-up
- Baseten · 7.99/10
Scores higher than the pick on: Price / performance, Latency / reliability, Developer & agent DX, Lock-in risk (higher = worse).
- When not to use the pick
- Don't want to operate GPUs — use Fireworks/Together serverless.
- Lock-in / migration
- Runner-up Baseten has lower stored lock-in risk (4.5 vs 6, 0–10, higher = more lock-in). Pick exit signals: self-hostable (provider tag `self-host`).
- Sourced price (usd_per_million_input_tokens)
Pick: no sourced price in this job's unit (no_sourced_price).
Cheapest matching SKU: Baseten GPT OSS 120B $0.1 — source, as of 2026-10-19, re-checked 2026-10-27. Not the pick; not quality-adjusted.
Price history: first seen 2026-10-19 · last changed no change recorded · history JSON
- Free option
- No free resource lists this job. A cheap paid SKU is not a free option.
- Failure modes to watch
- Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
Cite: Agent Provider Depot, Depot Pick for self-host-nim: nvidia-nim, scores as of 2026-10-19. Agents: GET /api/decide?jtbd=self-host-nim
All sourced prices, cheapest first
NVIDIA NIM / NGC
Capability of CUDA stacks + privacy. Note: vllm-note is conceptual — prefer NIM or Baseten if no vLLM provider entry.
Pricing (directional): As of 2026-10-24 (nvidia.com/en-us/ai-data-science/products/nim-microservices, fetched 2026-10-24; NVIDIA publishes no NIM price page, and this product page states no price figures, so production licence price stays null — 'Contact Sales'):…
Lock-in risk: higher = more lock-in (worse). Overall already inverts this dimension.
Choose when you self-host inference on NVIDIA GPUs and want blessed containers. For serverless tokens use Fireworks/Groq.
Quote for Self-hosted optimized model serving
Depot Pick stays NVIDIA NIM / NGC. Prices below are sourced SKU rows only (as of 2026-10-27). Input and output are not blended. GET /api/quote?jtbd=self-host-nim
Only 1 candidate has a sourced input $/1M; nothing was compared, so pickIsPriceWinner is null.
Sourced price unavailable for the Depot Pick. No row in the pricing index.
2 of 4 candidates have no sourced price row. 3 are excluded from the Input $/1M comparison (chat_completion_input). min sourced input SKU among candidates; not quality-adjusted
Runners-up
Baseten
Strong custom-model serving alternative to Replicate/Modal when you bring your own model.
Fireworks AI
Top pick for production open-model serving with real DX. Prefer over random GPU DIY when you want tokens not…
Replicate
Ship a model demo by lunch. Migrate hot paths to Modal/RunPod/Fireworks when the bill hurts.
When NOT to use the top pick
Don't want to operate GPUs — use Fireworks/Together serverless.
Full ranking (score table)
Ranked with this JTBD's weights. Scores are curated estimates unless marked otherwise. Agents: prefer /api/decide then this table for tie-breaks.
| # | Provider | Overall | Fit | $/perf | Latency | DX | As of |
|---|---|---|---|---|---|---|---|
| 1 | Fireworks AI | 8.3 | 8.6 | 8.8 | 8.4 | 9.0 | 2026-09-14 |
| 2 | Baseten | 8.0 | 8.4 | 7.8 | 8.4 | 8.6 | 2026-09-14 |
| 3 | Replicate | 7.8 | 8.4 | 6.0 | 8.2 | 9.4 | 2026-09-14 |
| 4 | NVIDIA NIM / NGCDepot Pick | 7.7 | 8.6 | 7.0 | 8.0 | 6.8 | 2026-10-19 |
Score methodology for this job
Capability of CUDA stacks + privacy. Note: vllm-note is conceptual — prefer NIM or Baseten if no vLLM provider entry.
capabilityFit: 25%pricePerformance: 18%latencyReliability: 15%developerDx: 15%privacyPosture: 10%lockInRisk: 7%maturity: 10%