AP
Agent Provider Depot
Opinionated picks for agents
OpenAPI

medium · bench-orch-008

Idempotent merge — multi-trial pass@3

Prompt

Orchestrate two workers writing to a shared notes doc with idempotent keys. pass@3: three trials must end with exactly one note per key and correct merged text.

Success criteria

  • No duplicate keys each trial
  • pass@3 = 1.0
  • Merged text matches gold

Scoring hints

  • reliabilityConsistency
  • Idempotency under parallel writes
Budget 1000s
Tools required · web no · code no
Tags fan-out, multi-trial, pass@3, merge

Primary type Multi-agent orchestrator. Gold is not in this payload. API

More multi_agent_orchestrator tasks