Trajectory SFT, DPO & GRPO Distillation (Teacher-to-Student Agent Fine-Tuning)
Distill verified high-scoring trajectories from an expensive frontier teacher model (plus rejected failure pairs) into a smaller, faster open-weight student model via LoRA SFT, Direct Preference Optimization (DPO), or Verifier-Guided GRPO.
At a glance — Oct 2026
Updated 2026-10-28 · forward this page: /jtbds/preference-dpo-sft-trace-distillation#briefPromotion class: deterministic_source_reverified · 8.3 ± 0.5 (7.8–8.8 · editorial_prior) · Pricing verified 2026-10-03.
- Depot Pick
- Together AI · 8.25/10 for this job (curated estimate, high confidence)
Together AI offers turnkey serverless and dedicated fine-tuning (LoRA SFT, DPO, and continued pretraining across Qwen, Llama, and DeepSeek families) with direct one-click deployment onto Together serverless endpoints, avoiding idle GPU burn while serving distilled agent adapters.
- Runner-up
- Fireworks AI · 8.40/10
Scores higher than the pick on: Capability fit, Latency / reliability, Developer & agent DX, Privacy / data retention.
- When not to use the pick
- Ultra-low-latency serverless LoRA multi-tenant deployment after tuning — Fireworks AI; experiment tracking, dataset lineage, and reward-curve telemetry across RL/DPO sweeps — Weights & Biases; raw multi-node H100/B200 clusters for custom Megatron/vLLM RL loops — Lambda or Modal.
- Lock-in / migration
- Pick exit signals: serves open-weight models (provider tag `open-models`).
- Sourced price (usd_per_million_train_tokens)
Pick: no sourced price in this job's unit (no_matching_sourced_price).
Cheapest matching SKU: Fireworks AI Managed training LoRA SFT up to 16B $0.5 — source, as of 2026-10-28, re-checked 2026-10-28. Not the pick; not quality-adjusted.
Price history: first seen 2026-10-03 · last changed no change recorded · history JSON
- Free option (runnable / substitutable)
- No runnable free tier lists this job. A cheap paid SKU or benchmark dataset is not a runnable free option.
- Failure modes to watch
- Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
Cite: Agent Provider Depot, Depot Pick for preference-dpo-sft-trace-distillation: together (editorial prior, scores as of 2026-10-28). Agents: GET /api/decide?jtbd=preference-dpo-sft-trace-distillation
All sourced prices, cheapest first
Together AI
Together AI offers turnkey serverless and dedicated fine-tuning (LoRA SFT, DPO, and continued pretraining across Qwen, Llama, and DeepSeek families) with direct one-click deployment onto Together serverless endpoints, avoiding idle GPU burn while serving distilled agent adapters.
Pricing (directional): As of 2026-10-28: Open-model inference + fine-tune on one OpenAI-compatible API; competitive per-token rates (often ~$0.15–$0.90/MTok class depending on model). Strong for Llama/Qwen/DeepSeek families; Dedicated endpoints available.
Top pick (8.25) and #2 runner-up (8.4) are separated by 0.15 points (<= 0.35 weight-sensitivity band). Treat them as co-picks and use the workload tie-breaker rule below rather than the raw composite decimal.
Workload Tie-Breaker Rule: Ultra-low-latency serverless LoRA multi-tenant deployment after tuning — Fireworks AI; experiment tracking, dataset lineage, and reward-curve telemetry across RL/DPO sweeps — Weights & Biases; raw multi-node H100/B200 clusters for custom Megatron/vLLM RL loops — Lambda or Modal.
Lock-in risk: higher = more lock-in (worse). Overall already inverts this dimension.
Best catalog breadth for OSS models. Watch minimums and lifetime spend gates before you design around a gated model.
Quote for Trajectory SFT, DPO & GRPO Distillation (Teacher-to-Student Agent Fine-Tuning)
Depot Pick stays Together AI. Prices below are sourced SKU rows only (as of 2026-10-28). Input and output are not blended. GET /api/quote?jtbd=preference-dpo-sft-trace-distillation
Only 1 candidate has a matching Train $/1M; nothing was compared, so pickIsPriceWinner is null.
- Together Fine-Tuning: Llama 3.1 8B / Qwen3.5 9B LoRA SFT ($0.34 / 1M training tokens) usd_per_million_train_tokens $0.34 source as of 2026-10-28
- Together Fine-Tuning: Llama 3.3 70B LoRA SFT ($2.03 / 1M training tokens) usd_per_million_train_tokens $2.03 source as of 2026-10-28
1 of 4 candidates have no sourced price row. 3 are excluded from the Train $/1M comparison (train). min sourced training-token SKU among candidates; not chat inference input; not quality-adjusted
Runners-up
Fireworks AI
Top pick for production open-model serving with real DX. Prefer over random GPU DIY when you want tokens not…
Weights & Biases
Use for training/fine-tune discipline. For pure agent request tracing, Langfuse/Helicone are leaner.
Modal
Depot darling for Python agent backends and bursty GPU jobs. If you think in functions not VMs, start here.
When NOT to use the top pick
Ultra-low-latency serverless LoRA multi-tenant deployment after tuning — Fireworks AI; experiment tracking, dataset lineage, and reward-curve telemetry across RL/DPO sweeps — Weights & Biases; raw multi-node H100/B200 clusters for custom Megatron/vLLM RL loops — Lambda or Modal.
Full ranking (score table)
Ranked with this JTBD's weights. Scores are curated estimates unless marked otherwise. Agents: prefer /api/decide then this table for tie-breaks.
| # | Provider | Overall | Fit | $/perf | Latency | DX | Lock-In Risk | As of |
|---|---|---|---|---|---|---|---|---|
| 1 | Modal | 8.5 | 9.0 | 8.2 | 8.6 | 9.6 | 4.5 | 2026-10-28 |
| 2 | Fireworks AI | 8.4 | 8.6 | 8.8 | 8.4 | 9.0 | 3.5 | 2026-10-28 |
| 3 | Together AIDepot Pick | 8.3 | 8.5 | 9.0 | 8.0 | 8.4 | 3.5 | 2026-10-28 |
| 4 | Weights & Biases | 8.0 | 8.8 | 6.8 | 9.0 | 8.2 | 5.5 | 2026-10-28 |
Score methodology for this job
Distillation only helps when training pairs are filtered by a deterministic verifier (unit tests, schema validation, or Countersign receipt). Training on unverified teacher traces amplifies hallucinations. Always benchmark the distilled student against the teacher on `costPerSolvedTaskUsd` and `pass@1`.
capabilityFit: 28%pricePerformance: 20%latencyReliability: 14%developerDx: 16%privacyPosture: 10%lockInRisk: 6%maturity: 6%