Hosted open-weight model inference

Scores as of 2026-09-14 (43 days old; Depot marks scores stale after 30 days). The pick is still Depot's current editorial pick; it has not been re-reviewed since. Prices below carry their own dates.

Depot Pick

Fireworks AI · 8.35/10 for this job (curated estimate, high confidence)

Catalog freshness, tokens/sec, OpenAI-compat DX, and pricePerf.

Runner-up

Together AI · 8.20/10

Scores higher than the pick on: Price / performance, Maturity / track record.

When not to use the pick

Need Hub-native custom model endpoints (HF) or ultra-low latency on narrow SKUs (Groq).

Lock-in / migration

Pick exit signals: serves open-weight models (provider tag `open-models`). Self-host fallback listed for this job: Qwen3-0.6B-Base, DeepSeek-V3.2 weights, Gemma models under the Gemma Terms of Use.

Sourced price (usd_per_million_input_tokens)

Pick: no sourced price in this job's unit (no_matching_sourced_price).

Cheapest matching SKU: Chutes Qwen3.8-27B (Qwen/Qwen3.8-27B-TEE) $0.039 — source, as of 2026-10-27, re-checked 2026-10-27. Not the pick; not quality-adjusted.

Price history: first seen 2026-10-03 · last changed 2026-10-19 · history JSON

Free option

• DeepSeek-R1 weights (oss) — source, as of 2026-10-25
• DeepSeek-V3.2 weights (oss) — source, as of 2026-10-25
• exo (oss) — source, as of 2026-10-25

Failure modes to watch

• Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
• Context overflow: If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.
• Lost in the middle: Ask the agent to name the passage it used. Spot-check questions whose only support sits in the middle of the context, not only at the edges.
• Non-determinism: Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.
• Schema drift: Validate every tool call and every loaded table against the current schema. Quarantine mismatches instead of coercing them.
• Silent truncation: Tools must return whether the payload was truncated and how long the original was. Fail the task if the answer depends on the missing span.

Reported outcomes (submitted reports only)

This job: loading… JSON · reports never change scores · seed/test records excluded

Depot Pick (Fireworks AI) on this job: loading… JSON · reports never change scores · seed/test records excluded

Cite: Agent Provider Depot, Depot Pick for open-model-inference: fireworks, scores as of 2026-09-14. Agents: GET /api/decide?jtbd=open-model-inference