{"asOf":"2026-10-03","doesNotChangeDepotPicks":true,"doesNotChangePriceSort":true,"doesNotChangeFreeIndex":true,"note":"Classification and failure modes are mechanisms. Case studies are included only when the source URL was fetched on 2026-10-03. A capability ceiling is not a company incident. Vendor figures are labeled as that vendor's report.","id":"orchestrator","name":"Multi-agent orchestrator","definition":"A lead assigns subtasks to workers and accepts a result only when it matches the assignment.","needs":{"jtbdIds":["durable-agent-workflows","event-driven-agent-jobs","langgraph-deploy","ray-distributed-agents","a2a-agent-discovery","agent-tool-router","mcp-discovery"],"freeKinds":["free_mcp","dev_tool"],"capabilityTags":["mcp","agents"],"plain":["A written assignment per worker","A budget for the whole fan-out"]},"failureModeIds":["inter-agent-misalignment","tool-loop","runaway-spend","identity-confusion","partial-tool-success","weak-verification","instruction-drift","hallucinated-tool-args","observability-gap"],"exampleTasks":["Delegate two searches and reject a worker result that answers a different question","Stop the fan-out when the tool-call budget is spent"],"failures":[{"id":"inter-agent-misalignment","name":"Inter-agent misalignment","whatGoesWrong":"A worker follows a different task than the one the lead assigned, or the lead merges results that do not answer the assignment.","howToDetect":"Log the assigned subtask next to the worker's stated result. The lead rejects a result that does not match the assignment.","classIds":["orchestrator","research","eval-judge"],"sourceUrls":["https://arxiv.org/abs/2503.13657"]},{"id":"tool-loop","name":"Tool loop","whatGoesWrong":"The agent calls the same tool, or a cycle of tools, without new information, so the task never reaches a stop condition.","howToDetect":"Hash the last several tool names and arguments. Stop when the same call repeats and the observation did not change. Cap tool calls per task.","classIds":["coding","research","browser","computer-use","orchestrator","scheduled-ops"],"sourceUrls":["https://www.anthropic.com/engineering/built-multi-agent-research-system"]},{"id":"runaway-spend","name":"Runaway spend","whatGoesWrong":"Retries, fan-out, or a loop keep calling paid models or tools after the task should have stopped.","howToDetect":"Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.","classIds":["coding","research","orchestrator","scheduled-ops","sales-outreach","voice"],"sourceUrls":["https://www.anthropic.com/engineering/building-effective-agents"]},{"id":"identity-confusion","name":"Identity confusion","whatGoesWrong":"The agent acts for the wrong user or tenant, or a principal treats the agent as a separate party that the operator does not have to answer for.","howToDetect":"Bind every tool call to a user or tenant id. Refuse the call if that id changed mid-task. For a public bot, the operator owns the words the bot shows.","classIds":["support","personal-assistant","sales-outreach","orchestrator","voice"],"sourceUrls":["https://decisions.civilresolutionbc.ca/crt/crtd/en/525448/1/document.do"]},{"id":"partial-tool-success","name":"Partial tool success","whatGoesWrong":"A tool completes only part of a batch and the agent treats the whole batch as done.","howToDetect":"Require a per-item status and a failure count. Success for the task requires the failed count to be zero or explicitly handed to a retry queue.","classIds":["coding","data-etl","browser","computer-use","voice","orchestrator","scheduled-ops"],"sourceUrls":[]},{"id":"weak-verification","name":"Weak verification","whatGoesWrong":"The run is marked successful because the agent said it was done, without an execution check, a test, or a second pass.","howToDetect":"Success requires an end-state check the agent does not grade for itself: tests, a database diff, or a separate judge that cannot see the gold label.","classIds":["coding","eval-judge","orchestrator","support","security-review"],"sourceUrls":["https://arxiv.org/abs/2503.13657","https://arxiv.org/abs/2310.06770"]},{"id":"instruction-drift","name":"Instruction drift","whatGoesWrong":"Over a long trace the agent stops following the user's rule or the stop condition, including continuing after it already has enough information.","howToDetect":"Restate the stop condition in the trace. End the run when it is met. Review long traces for steps that do not serve the original task.","classIds":["research","orchestrator","coding","browser","personal-assistant"],"sourceUrls":["https://www.anthropic.com/engineering/built-multi-agent-research-system","https://arxiv.org/abs/2308.03688"]},{"id":"hallucinated-tool-args","name":"Hallucinated tool arguments","whatGoesWrong":"The agent calls a tool with an id, path, or field that does not exist.","howToDetect":"Reject a call that fails the current schema or names an unknown id, and do that before the tool runs a side effect.","classIds":["coding","data-etl","browser","orchestrator"],"sourceUrls":["https://arxiv.org/abs/2308.03688"]},{"id":"observability-gap","name":"Observability gap","whatGoesWrong":"Traces or run ingestion stop, and the failure is outside the agent's reasoning. A certificate or DNS fault can make the trace API fail while agents keep acting.","howToDetect":"Alert on a drop in trace ingestion and on certificate expiry for the trace host. Do not treat a missing trace as a successful quiet run.","classIds":["eval-judge","scheduled-ops","orchestrator"],"sourceUrls":["https://www.langchain.com/blog/langsmith-incident-on-may-1-2025"]}],"caseStudies":[{"id":"anthropic-building-effective-agents-2024","title":"Building effective agents","year":2024,"studyKind":"vendor_writeup","classIds":["orchestrator","coding","research"],"failureModeIds":["runaway-spend","instruction-drift"],"lesson":"The post separates workflows, where the path is orchestrated in code, from agents, where the model decides the path. It says agentic systems often trade latency and cost for task performance, and that the successful implementations they saw used simple composable patterns. A note on the fetched page says much of the tooling landscape described has changed since December 2024.","sourceUrl":"https://www.anthropic.com/engineering/building-effective-agents","asOf":"2026-10-03","notes":"Fetched page says Published Dec 19, 2024. This is a design note, not an incident with a measured outage."},{"id":"anthropic-multi-agent-research-2025","title":"How we built our multi-agent research system","year":2025,"studyKind":"vendor_writeup","classIds":["research","orchestrator"],"failureModeIds":["instruction-drift","tool-loop","inter-agent-misalignment"],"lesson":"The post describes an orchestrator-worker research system. Watching traces showed agents continuing after they already had enough results, using overly verbose search queries, or selecting the wrong tool. It also says parallel tool use cut research time by up to 90% for complex queries in their system. That 90% figure is Anthropic's report about their Research feature, not a Depot measurement.","sourceUrl":"https://www.anthropic.com/engineering/built-multi-agent-research-system","asOf":"2026-10-03","notes":"Fetched page says Published Jun 13, 2025."},{"id":"the-agent-company-2025","title":"TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks","year":2025,"studyKind":"capability_ceiling","classIds":["coding","browser","research","orchestrator"],"failureModeIds":["weak-verification"],"lesson":"The abstract describes a simulated software company where agents browse, write code, run programs, and communicate with coworkers. It says the most competitive agent they tested completed 30% of tasks autonomously. Read that as a ceiling inside their simulation, not as a measurement of any employer's production team.","sourceUrl":"https://arxiv.org/abs/2412.14161","asOf":"2026-10-03","notes":"Fetched arXiv v3, last revised 10 Sep 2025. Submitted 18 Dec 2024."},{"id":"mast-multi-agent-failures-2025","title":"Why Do Multi-Agent LLM Systems Fail?","year":2025,"studyKind":"mechanism_study","classIds":["orchestrator","eval-judge"],"failureModeIds":["inter-agent-misalignment","weak-verification"],"lesson":"The abstract says performance gains of multi-agent systems on popular benchmarks are often minimal. Their taxonomy, built from 150 traces with annotator kappa of 0.88, groups failures into system design, inter-agent misalignment, and task verification, and identifies 14 modes. The larger MAST-Data collection is described as 1600+ annotated traces. This is a failure taxonomy, not a single production outage.","sourceUrl":"https://arxiv.org/abs/2503.13657","asOf":"2026-10-03","notes":"Fetched the arXiv abstract (submitted 2025). The 14 mode names are not copied because the abstract did not list them."}]}