85 tasks · research ingest · original Depot designs
Tasks
Inspired by SWE-bench / WebArena / τ-bench / GAIA vibes — not copies. Gold stays server-side. Holdout tasks never return answers. JSON
| Task | Type | Diff | Budget |
|---|---|---|---|
| Patch CVE-style path traversal in file server bench-coding-005 | CodingOps / SRE | adversarial | 20m |
| Find error-rate spike cause from logs bench-ops-001 | Ops / SRE | easy | 10m |
| Runbook: restart with change freeze bench-ops-002 | Ops / SRE | medium | 10m |
| Kubernetes CrashLoop diagnosis bench-ops-003 | Ops / SRE | hard | 25m |
| Prompt injection via ticket comment bench-ops-004 | Ops / SRE | adversarial | 10m |
| Rollback decision under partial metrics bench-ops-005 | Ops / SRE | medium | 12m |
| Browser+ops: status page incident note bench-cross-003 | BrowserOps / SRE | medium | 10m |
| Error spike RCA — multi-trial pass@3 bench-ops-006 | Ops / SRE | medium | 15m |
| Cascading tool failure during incident bench-ops-007 | Ops / SRE | hard | 20m |
| Conflicting freeze vs SEV-1 page bench-ops-008 | Ops / SRE | adversarial | 10m |
| General+ops: incident tweet vs internal truth bench-cross-010 | GeneralOps / SRE | adversarial | 10m |
11 tasks