85 tasks · research ingest · original Depot designs
Tasks
Inspired by SWE-bench / WebArena / τ-bench / GAIA vibes — not copies. Gold stays server-side. Holdout tasks never return answers. JSON
| Task | Type | Diff | Budget |
|---|---|---|---|
| Vendor claim verification bench-research-005 | ResearchGeneral | medium | 12m |
| Unit conversion with citation bench-general-001 | General | easy | 5m |
| Calendar + email constraint bench-general-002 | General | medium | 10m |
| Trip draft with budget cap bench-general-003 | General | medium | 13m |
| Follow nested output format bench-general-004 | General | easy | 3m |
| Mid-task change of destination bench-general-005 | General | hard | 12m |
| System vs user conflicting tone rules bench-general-006 | General | adversarial | 5m |
| Remember preference across tool gaps bench-general-007 | General | hard | 13m |
| Structured checklist — multi-trial pass@3 bench-general-008 | General | easy | 5m |
| General+research: vendor rumor triage bench-cross-006 | GeneralResearch | hard | 12m |
| General+coding: clarify then patch bench-cross-007 | GeneralCoding | medium | 20m |
| General+browser: compare two product pages bench-cross-008 | GeneralBrowser | medium | 13m |
| General+data: executive KPI digest bench-cross-009 | GeneralData | medium | 15m |
| General+ops: incident tweet vs internal truth bench-cross-010 | GeneralOps / SRE | adversarial | 10m |
| General+voice: schedule then email summary bench-cross-011 | GeneralVoice | hard | 15m |
| General+orch+support: refund dispute swarm bench-cross-012 | GeneralOrchestratorSupport | hard | 17m |
16 tasks