{"asOf":"2026-10-03","doesNotChangeDepotPicks":true,"doesNotChangePriceSort":true,"doesNotChangeFreeIndex":true,"note":"Classification and failure modes are mechanisms. Case studies are included only when the source URL was fetched on 2026-10-03. A capability ceiling is not a company incident. Vendor figures are labeled as that vendor's report.","id":"research","name":"Research agent","definition":"Gathers sources for an open-ended question and returns an answer that names those sources.","needs":{"jtbdIds":["deep-research-agents","web-grounded-qa","agent-search","search-augmented-llm","web-scrape","llm-ready-extract"],"freeKinds":["free_docs","free_data","free_browser"],"capabilityTags":["search","agents"],"plain":["A stop rule for when enough sources have been read"]},"failureModeIds":["prompt-injection","stale-retrieval","tool-loop","runaway-spend","lost-in-the-middle","ungrounded-answer","instruction-drift","inter-agent-misalignment","context-overflow","policy-conflict","silent-truncation"],"exampleTasks":["Answer a question with the URLs that support each claim","Say when the fetched pages do not contain the answer","Split a question across workers and check that each result matches its assignment"],"failures":[{"id":"prompt-injection","name":"Prompt injection","whatGoesWrong":"Untrusted text, including text from a page, file, or message the agent fetched, changes the agent's behavior away from the user's task. OWASP distinguishes a direct injection in the user's own prompt from an indirect injection in outside content. Neither form is fully removed by retrieval or fine-tuning, according to the OWASP LLM01:2025 page fetched today.","howToDetect":"Compare each tool call to the original user task. If the call only makes sense given text that arrived from a fetched page, file, or message, stop and ask. Record which source the instruction came from. This catalog does not store payloads or attack steps.","classIds":["browser","computer-use","research","support","personal-assistant","security-review","sales-outreach"],"sourceUrls":["https://genai.owasp.org/llmrisk/llm01-prompt-injection/","https://www.anthropic.com/news/developing-computer-use","https://simonwillison.net/2023/Apr/14/worst-that-can-happen/"]},{"id":"stale-retrieval","name":"Stale retrieval","whatGoesWrong":"The answer is grounded in a passage, index, or policy page that is older than the source the operator now treats as canonical.","howToDetect":"Store the source URL and the time it was fetched. If a newer canonical page exists, or two stored sources disagree, do not answer from the older one.","classIds":["rag","research","support","data-etl","personal-assistant","sales-outreach","local-offline"],"sourceUrls":["https://decisions.civilresolutionbc.ca/crt/crtd/en/525448/1/document.do"]},{"id":"tool-loop","name":"Tool loop","whatGoesWrong":"The agent calls the same tool, or a cycle of tools, without new information, so the task never reaches a stop condition.","howToDetect":"Hash the last several tool names and arguments. Stop when the same call repeats and the observation did not change. Cap tool calls per task.","classIds":["coding","research","browser","computer-use","orchestrator","scheduled-ops"],"sourceUrls":["https://www.anthropic.com/engineering/built-multi-agent-research-system"]},{"id":"runaway-spend","name":"Runaway spend","whatGoesWrong":"Retries, fan-out, or a loop keep calling paid models or tools after the task should have stopped.","howToDetect":"Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.","classIds":["coding","research","orchestrator","scheduled-ops","sales-outreach","voice"],"sourceUrls":["https://www.anthropic.com/engineering/building-effective-agents"]},{"id":"lost-in-the-middle","name":"Lost in the middle","whatGoesWrong":"The needed passage is in the prompt, but the model uses the beginning or the end and misses the middle of a long context.","howToDetect":"Ask the agent to name the passage it used. Spot-check questions whose only support sits in the middle of the context, not only at the edges.","classIds":["rag","research","local-offline"],"sourceUrls":["https://arxiv.org/abs/2307.03172"]},{"id":"ungrounded-answer","name":"Ungrounded answer","whatGoesWrong":"The agent states a fact, a citation, or a policy with no retrieved support, or with a citation that does not contain the claim.","howToDetect":"Require a source URL or passage id on factual sentences. Drop or mark any sentence whose source does not contain the claim.","classIds":["rag","research","support","sales-outreach"],"sourceUrls":["https://www.langchain.com/blog/rebuilding-chat-langchain"]},{"id":"instruction-drift","name":"Instruction drift","whatGoesWrong":"Over a long trace the agent stops following the user's rule or the stop condition, including continuing after it already has enough information.","howToDetect":"Restate the stop condition in the trace. End the run when it is met. Review long traces for steps that do not serve the original task.","classIds":["research","orchestrator","coding","browser","personal-assistant"],"sourceUrls":["https://www.anthropic.com/engineering/built-multi-agent-research-system","https://arxiv.org/abs/2308.03688"]},{"id":"inter-agent-misalignment","name":"Inter-agent misalignment","whatGoesWrong":"A worker follows a different task than the one the lead assigned, or the lead merges results that do not answer the assignment.","howToDetect":"Log the assigned subtask next to the worker's stated result. The lead rejects a result that does not match the assignment.","classIds":["orchestrator","research","eval-judge"],"sourceUrls":["https://arxiv.org/abs/2503.13657"]},{"id":"context-overflow","name":"Context overflow","whatGoesWrong":"The document or the session does not fit the context window, and the agent answers from whatever fragment was left in view.","howToDetect":"If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.","classIds":["rag","personal-assistant","research","local-offline","coding"],"sourceUrls":["https://arxiv.org/abs/2310.08560"]},{"id":"policy-conflict","name":"Policy conflict","whatGoesWrong":"Two sources the operator owns disagree, and the agent states one of them as the policy.","howToDetect":"If cited sources contradict each other, return the conflict and the URLs. Do not pick a winner silently.","classIds":["support","sales-outreach","rag","research"],"sourceUrls":["https://decisions.civilresolutionbc.ca/crt/crtd/en/525448/1/document.do"]},{"id":"silent-truncation","name":"Silent truncation","whatGoesWrong":"Tool output or a document is cut off without a marker, and the agent answers as if it saw the whole thing.","howToDetect":"Tools must return whether the payload was truncated and how long the original was. Fail the task if the answer depends on the missing span.","classIds":["rag","coding","data-etl","voice","local-offline","research"],"sourceUrls":["https://arxiv.org/abs/2310.08560"]}],"caseStudies":[{"id":"gaia-2023","title":"GAIA: a benchmark for General AI Assistants","year":2023,"studyKind":"capability_ceiling","classIds":["research","personal-assistant"],"failureModeIds":["weak-verification"],"lesson":"The abstract says GAIA questions need reasoning, multi-modal handling, web browsing, and tool use, and that they are conceptually simple for people. It reports human respondents at 92% and GPT-4 with plugins at 15%, on a set of 466 questions. That gap is a ceiling for a general assistant, not an incident at a company.","sourceUrl":"https://arxiv.org/abs/2311.12983","asOf":"2026-10-03","notes":"Fetched the arXiv abstract (submitted 21 Nov 2023). Answers for 300 questions were held for a leaderboard, so a public question file is not the full scored set."},{"id":"anthropic-building-effective-agents-2024","title":"Building effective agents","year":2024,"studyKind":"vendor_writeup","classIds":["orchestrator","coding","research"],"failureModeIds":["runaway-spend","instruction-drift"],"lesson":"The post separates workflows, where the path is orchestrated in code, from agents, where the model decides the path. It says agentic systems often trade latency and cost for task performance, and that the successful implementations they saw used simple composable patterns. A note on the fetched page says much of the tooling landscape described has changed since December 2024.","sourceUrl":"https://www.anthropic.com/engineering/building-effective-agents","asOf":"2026-10-03","notes":"Fetched page says Published Dec 19, 2024. This is a design note, not an incident with a measured outage."},{"id":"anthropic-multi-agent-research-2025","title":"How we built our multi-agent research system","year":2025,"studyKind":"vendor_writeup","classIds":["research","orchestrator"],"failureModeIds":["instruction-drift","tool-loop","inter-agent-misalignment"],"lesson":"The post describes an orchestrator-worker research system. Watching traces showed agents continuing after they already had enough results, using overly verbose search queries, or selecting the wrong tool. It also says parallel tool use cut research time by up to 90% for complex queries in their system. That 90% figure is Anthropic's report about their Research feature, not a Depot measurement.","sourceUrl":"https://www.anthropic.com/engineering/built-multi-agent-research-system","asOf":"2026-10-03","notes":"Fetched page says Published Jun 13, 2025."},{"id":"the-agent-company-2025","title":"TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks","year":2025,"studyKind":"capability_ceiling","classIds":["coding","browser","research","orchestrator"],"failureModeIds":["weak-verification"],"lesson":"The abstract describes a simulated software company where agents browse, write code, run programs, and communicate with coworkers. It says the most competitive agent they tested completed 30% of tasks autonomously. Read that as a ceiling inside their simulation, not as a measurement of any employer's production team.","sourceUrl":"https://arxiv.org/abs/2412.14161","asOf":"2026-10-03","notes":"Fetched arXiv v3, last revised 10 Sep 2025. Submitted 18 Dec 2024."},{"id":"lost-in-the-middle-2023","title":"Lost in the Middle: How Language Models Use Long Contexts","year":2023,"studyKind":"mechanism_study","classIds":["rag","research"],"failureModeIds":["lost-in-the-middle"],"lesson":"The abstract says that on multi-document question answering and key-value retrieval, performance is often highest when the relevant information is at the beginning or the end of the context, and degrades when it is in the middle, including for models billed as long-context. Putting the best passage in the middle of a prompt is not the same as the model using it.","sourceUrl":"https://arxiv.org/abs/2307.03172","asOf":"2026-10-03","notes":"Fetched the arXiv abstract. Comment: accepted to TACL 2023. Not a vendor incident."}]}