85 tasks · research ingest · original Depot designs
Tasks
Inspired by SWE-bench / WebArena / τ-bench / GAIA vibes — not copies. Gold stays server-side. Holdout tasks never return answers. JSON
| Task | Type | Diff | Budget |
|---|---|---|---|
| Multi-hop factual short answer bench-research-001 | Research | easy | 10m |
| Synthesize conflicting sources bench-research-002 | Research | medium | 15m |
| Unknowable claim abstention bench-research-003 | Research | adversarial | 5m |
| Literature map with citation graph bench-research-004 | Research | hard | 30m |
| Vendor claim verification bench-research-005 | ResearchGeneral | medium | 12m |
| Code+research: implement from paper abstract bench-cross-001 | CodingResearch | hard | 25m |
| Orchestrator+research adversarial decoy bench-cross-005 | OrchestratorResearch | adversarial | 13m |
| Hallucination trap: plausible fake DOI bench-research-006 | Research | adversarial | 7m |
| Source conflict with recency constraint bench-research-007 | Research | hard | 15m |
| Multi-hop company fact — multi-trial pass@3 bench-research-008 | Research | medium | 13m |
| General+research: vendor rumor triage bench-cross-006 | GeneralResearch | hard | 12m |
11 tasks