Confidential / TEE-backed LLM inference
Run inference where the host cannot read prompts/weights in cleartext — confidential VMs or TEE-backed serverless for regulated or high-trust agents.
At a glance — Oct 2026
Updated 2026-10-27 · forward this page: /jtbds/confidential-tee-inference#briefScores as of 2026-09-14 (43 days old; Depot marks scores stale after 30 days). The pick is still Depot's current editorial pick; it has not been re-reviewed since. Prices below carry their own dates.
- Depot Pick
- Chutes · 6.82/10 for this job (curated estimate, high confidence)
Capability = real TEE/confidential posture + model coverage. Privacy weighted high. Chutes publishes a TEE matrix agents can pin; hyperscalers win when enterprise paperwork dominates. Confidence: medium — re-curl chutes.ai/pricing TEE rows before production.
- Runner-up
- FriendliAI · 8.01/10
Scores higher than the pick on: Capability fit, Latency / reliability, Developer & agent DX, Privacy / data retention, Maturity / track record.
- When not to use the pick
- Need hyperscaler confidential-compute contracts with BAA/DPA already negotiated (Bedrock/Azure), or a dedicated Friendli/GPU TEE SKU your security team already approved.
- Lock-in / migration
- Pick exit signals: serves open-weight models (provider tag `open-model`).
- Sourced price (usd_per_million_input_tokens)
Pick's lowest sourced SKU: Qwen3.8-27B (Qwen/Qwen3.8-27B-TEE) $0.039 (usd_per_million_input_tokens) — source, as of 2026-10-27, re-checked 2026-10-27
Cheapest matching SKU: Chutes Qwen3.8-27B (Qwen/Qwen3.8-27B-TEE) $0.039 — source, as of 2026-10-27, re-checked 2026-10-27. The pick is the cheapest.
Price history: first seen 2026-10-03 · last changed 2026-10-19 · history JSON
- Free option
- No free resource lists this job. A cheap paid SKU is not a free option.
- Failure modes to watch
- Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
Same-job, same-unit comparisons: Chutes vs FriendliAI · Chutes vs Amazon Bedrock
Cite: Agent Provider Depot, Depot Pick for confidential-tee-inference: chutes, scores as of 2026-09-14. Agents: GET /api/decide?jtbd=confidential-tee-inference
All sourced prices, cheapest first
Chutes
Capability = real TEE/confidential posture + model coverage. Privacy weighted high. Chutes publishes a TEE matrix agents can pin; hyperscalers win when enterprise paperwork dominates. Confidence: medium — re-curl chutes.ai/pricing TEE rows before production.
Pricing (directional): As of 2026-10-27 (chutes.ai/pricing, fetched twice 2026-10-27): Public TEE inference PAYG per 1M tokens, input/output (the page says per-model rates are 'pulled live from our pricing API'): gemma-4-31B-turbo $0.12/$0.37; Qwen3.8-27B $0.039/…
Lock-in risk: higher = more lock-in (worse). Overall already inverts this dimension.
Budget open-model overflow with a now-pinned public TEE matrix. Depot open-model production pick stays Fireworks; ultra-budget volume DeepInfra/Novita; use Chutes as capacity valve and re-check live rates.
Quote for Confidential / TEE-backed LLM inference
Depot Pick stays Chutes. Prices below are sourced SKU rows only (as of 2026-10-27). Input and output are not blended. GET /api/quote?jtbd=confidential-tee-inference
Pick is the cheapest sourced input $/1M among candidates (0.039).
- DeepSeek-V3.2 (deepseek-ai/DeepSeek-V3.2-TEE) usd_per_million_input_tokens $1 source as of 2026-10-27
- DeepSeek-V4-Flash-0731 (deepseek-ai/DeepSeek-V4-Flash-0731-TEE) usd_per_million_input_tokens $0.44 source as of 2026-10-27
- gemma-4-31B-turbo (google/gemma-4-31B-turbo-TEE) usd_per_million_input_tokens $0.12 source as of 2026-10-27
- GLM-5.1 (zai-org/GLM-5.1-TEE) usd_per_million_input_tokens $0.9 source as of 2026-10-27
- GLM-5.2 (zai-org/GLM-5.2-TEE) usd_per_million_input_tokens $1.25 source as of 2026-10-27
- Kimi K2.6 (moonshotai/Kimi-K2.6-TEE) usd_per_million_input_tokens $0.39 source as of 2026-10-27
- Kimi-K3 (moonshotai/Kimi-K3-TEE) usd_per_million_input_tokens $1.85 source as of 2026-10-27
- Mistral-Nemo-Instruct-2407 (unsloth/Mistral-Nemo-Instruct-2407-TEE) usd_per_million_input_tokens $0.0245 source as of 2026-10-03
- Nemotron-3-Nano-Omni-30B (Nemotron-3-Nano-Omni-30B-TEE) usd_per_million_input_tokens $0.0245 source as of 2026-10-03
- Qwen 3.5 (Qwen/Qwen3.5-397B-A17B-TEE) usd_per_million_input_tokens $0.45 source as of 2026-10-27
- Qwen3-235B-A22B-Thinking-2507 (Qwen/Qwen3-235B-A22B-Thinking-2507-TEE) usd_per_million_input_tokens $0.2989 source as of 2026-10-03
- Qwen3-32B (Qwen/Qwen3-32B-TEE) usd_per_million_input_tokens $0.104 source as of 2026-10-03
- Qwen3.6-27B (Qwen/Qwen3.6-27B-TEE) usd_per_million_input_tokens $0.25 source as of 2026-10-27
- Qwen3.8-27B (Qwen/Qwen3.8-27B-TEE) usd_per_million_input_tokens $0.039 source as of 2026-10-27
- DeepSeek-V3.2 (deepseek-ai/DeepSeek-V3.2-TEE) usd_per_million_output_tokens $1 source as of 2026-10-27
- DeepSeek-V4-Flash-0731 (deepseek-ai/DeepSeek-V4-Flash-0731-TEE) usd_per_million_output_tokens $1.32 source as of 2026-10-27
- gemma-4-31B-turbo (google/gemma-4-31B-turbo-TEE) usd_per_million_output_tokens $0.37 source as of 2026-10-27
- GLM-5.1 (zai-org/GLM-5.1-TEE) usd_per_million_output_tokens $2.9 source as of 2026-10-27
- GLM-5.2 (zai-org/GLM-5.2-TEE) usd_per_million_output_tokens $3.95 source as of 2026-10-27
- Kimi K2.6 (moonshotai/Kimi-K2.6-TEE) usd_per_million_output_tokens $1.8 source as of 2026-10-27
- Kimi-K3 (moonshotai/Kimi-K3-TEE) usd_per_million_output_tokens $9.5 source as of 2026-10-27
- Mistral-Nemo-Instruct-2407 (unsloth/Mistral-Nemo-Instruct-2407-TEE) usd_per_million_output_tokens $0.0978 source as of 2026-10-03
- Nemotron-3-Nano-Omni-30B (Nemotron-3-Nano-Omni-30B-TEE) usd_per_million_output_tokens $0.0978 source as of 2026-10-03
- Qwen 3.5 (Qwen/Qwen3.5-397B-A17B-TEE) usd_per_million_output_tokens $3 source as of 2026-10-27
- Qwen3-235B-A22B-Thinking-2507 (Qwen/Qwen3-235B-A22B-Thinking-2507-TEE) usd_per_million_output_tokens $1.1957 source as of 2026-10-03
- Qwen3-32B (Qwen/Qwen3-32B-TEE) usd_per_million_output_tokens $0.416 source as of 2026-10-03
- Qwen3.6-27B (Qwen/Qwen3.6-27B-TEE) usd_per_million_output_tokens $2 source as of 2026-10-27
- Qwen3.8-27B (Qwen/Qwen3.8-27B-TEE) usd_per_million_output_tokens $1.25 source as of 2026-10-27
- Plus usd_per_month $10 source as of 2026-10-03
- Private chute deployment fee usd_one_time $5.4 source as of 2026-10-27
- Pro usd_per_month $20 source as of 2026-10-03
- RTX Pro 6000 96GB usd_per_gpu_hour $1.8 source as of 2026-10-27
1 of 5 candidates have no sourced price row. 2 are excluded from the Input $/1M comparison (chat_completion_input). min sourced input SKU among candidates; not quality-adjusted
Runners-up
FriendliAI
Open-model production peer when latency/tool-calling matter. Fireworks remains depot open-model default…
Amazon Bedrock
Enterprise multi-model switchboard. Great when AWS is mandatory; mediocre when you just want the best Claude…
Azure OpenAI
Pick when IT already standardized on Azure and needs VNET/private endpoints. Otherwise go direct OpenAI and…
Fireworks AI
Top pick for production open-model serving with real DX. Prefer over random GPU DIY when you want tokens not…
When NOT to use the top pick
Need hyperscaler confidential-compute contracts with BAA/DPA already negotiated (Bedrock/Azure), or a dedicated Friendli/GPU TEE SKU your security team already approved.
Full ranking (score table)
Ranked with this JTBD's weights. Scores are curated estimates unless marked otherwise. Agents: prefer /api/decide then this table for tie-breaks.
| # | Provider | Overall | Fit | $/perf | Latency | DX | As of |
|---|---|---|---|---|---|---|---|
| 1 | Fireworks AI | 8.1 | 8.6 | 8.8 | 8.4 | 9.0 | 2026-09-14 |
| 2 | Amazon Bedrock | 8.0 | 8.8 | 6.0 | 9.2 | 6.8 | 2026-09-14 |
| 3 | FriendliAI | 8.0 | 8.4 | 8.0 | 8.4 | 8.2 | 2026-10-01 |
| 4 | Azure OpenAI | 7.9 | 9.0 | 5.5 | 9.0 | 7.0 | 2026-09-14 |
| 5 | ChutesDepot Pick | 6.8 | 7.4 | 9.2 | 6.8 | 7.6 | 2026-09-14 |
Score methodology for this job
Capability = real TEE/confidential posture + model coverage. Privacy weighted high. Chutes publishes a TEE matrix agents can pin; hyperscalers win when enterprise paperwork dominates. Confidence: medium — re-curl chutes.ai/pricing TEE rows before production.
capabilityFit: 20%pricePerformance: 10%latencyReliability: 12%developerDx: 10%privacyPosture: 30%lockInRisk: 8%maturity: 10%