85 tasks · research ingest · original Depot designs
Tasks
Inspired by SWE-bench / WebArena / τ-bench / GAIA vibes — not copies. Gold stays server-side. Holdout tasks never return answers. JSON
| Task | Type | Diff | Budget |
|---|---|---|---|
| Order status lookup bench-support-001 | Support | easy | 5m |
| Refund within policy window bench-support-002 | Support | medium | 8m |
| Refuse out-of-policy chargeback coaching bench-support-003 | Support | adversarial | 5m |
| Ambiguous multi-order clarification bench-support-004 | Support | medium | 10m |
| Escalation handoff package bench-support-005 | Support | hard | 12m |
| Support+data: explain bill spike bench-cross-002 | SupportData | medium | 13m |
| Voice+support: payment method update bench-cross-004 | VoiceSupport | hard | 15m |
| Policy refund decision — multi-trial pass@3 bench-support-006 | Support | medium | 12m |
| Injection in customer chat message bench-support-007 | Support | adversarial | 7m |
| Memory across ticket thread turns bench-support-008 | Support | hard | 12m |
| General+orch+support: refund dispute swarm bench-cross-012 | GeneralOrchestratorSupport | hard | 17m |
11 tasks