Air-gapped / VPC model serving

Scores as of 2026-10-19 (8 days old).

Depot Pick

NVIDIA NIM / NGC · 7.83/10 for this job (curated estimate, medium confidence)

Remove invalid runner if any. NIM for optimized self-host. Confidence: medium on packaging/licensing.

Runner-up

Hugging Face Inference · 7.93/10

Scores higher than the pick on: Price / performance, Developer & agent DX, Lock-in risk (higher = worse), Maturity / track record.

When not to use the pick

Private Bedrock/Azure endpoints suffice without NIM packaging, or HF Inference Endpoints already in VPC.

Lock-in / migration

Runner-up Hugging Face Inference has lower stored lock-in risk (3.5 vs 6, 0–10, higher = more lock-in). Pick exit signals: self-hostable (provider tag `self-host`).

Sourced price (usd_per_million_input_tokens)

Pick: no sourced price in this job's unit (no_sourced_price).

Cheapest matching SKU: Amazon Bedrock Gemma 3 4B $0.04 — source, as of 2026-10-03, re-checked 2026-10-27. Not the pick; not quality-adjusted.

Price history: first seen 2026-10-03 · last changed no change recorded · history JSON

Free option

No free resource lists this job. A cheap paid SKU is not a free option.

Failure modes to watch

• Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
• Context overflow: If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.
• Lost in the middle: Ask the agent to name the passage it used. Spot-check questions whose only support sits in the middle of the context, not only at the edges.
• Non-determinism: Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.
• Schema drift: Validate every tool call and every loaded table against the current schema. Quarantine mismatches instead of coercing them.
• Silent truncation: Tools must return whether the payload was truncated and how long the original was. Fail the task if the answer depends on the missing span.

Reported outcomes (submitted reports only)

This job: loading… JSON · reports never change scores · seed/test records excluded

Depot Pick (NVIDIA NIM / NGC) on this job: loading… JSON · reports never change scores · seed/test records excluded

Cite: Agent Provider Depot, Depot Pick for air-gapped-model-serve: nvidia-nim, scores as of 2026-10-19. Agents: GET /api/decide?jtbd=air-gapped-model-serve