{"asOf":"2026-10-03","doesNotChangeDepotPicks":true,"doesNotChangePriceSort":true,"doesNotChangeFreeIndex":true,"note":"Classification and failure modes are mechanisms. Case studies are included only when the source URL was fetched on 2026-10-03. A capability ceiling is not a company incident. Vendor figures are labeled as that vendor's report.","count":2,"catalogCount":27,"refused":["Replit agent database story (July 2025): The Register URL returned HTTP 403 and no first-party engineering postmortem was fetched. Not indexed.","CanLII HTML for 2024 BCCRT 149 returned a client-side block. The Civil Resolution Tribunal document was fetched instead and is the indexed source.","Cursor agent best-practices post loaded and is a product guide, not a postmortem. Not indexed as an incident. The University of Chicago study it mentions was not fetched.","Cognition 'Introducing Devin' (published 2024-03-12) duplicates the SWE-bench technical report. The report is indexed; the launch post is not a second study.","Anthropic many-shot jailbreaking research page loaded. It is an attack writeup, so it is not indexed and no procedure is stored.","OWASP LLM01:2025 was fetched and is cited on the prompt-injection failure mode only. Attack examples from that page are not copied into a case study.","Calder Lab and similar unsourced production-metric blogs were not fetched and are not indexed.","Google Research abstract for Data Cascades in High-Stakes AI loaded on 2026-10-03 (the abstract says 92% prevalence). Not indexed for data-etl: it is an interview study of ML data practice, not a pipeline job failure. The indexed data-etl study is the dbt Cloud status page."],"studies":[{"id":"tau-bench-2024","title":"τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains","year":2024,"studyKind":"capability_ceiling","classIds":["support","voice"],"failureModeIds":["non-determinism","policy-conflict","weak-verification"],"lesson":"The abstract says τ-bench checks whether an agent follows domain rules while talking to a simulated user, and grades the database at the end. It reports that even a function-calling agent such as gpt-4o succeeded on under 50% of tasks and was inconsistent, with pass^8 under 25% in the retail domain. One green trial is not reliability.","sourceUrl":"https://arxiv.org/abs/2406.12045","asOf":"2026-10-03","notes":"Fetched the arXiv abstract (submitted 17 Jun 2024). Capability ceiling for policy-following tool use, not a named company's outage."},{"id":"moffatt-v-air-canada-2024","title":"Moffatt v. Air Canada, 2024 BCCRT 149","year":2024,"studyKind":"tribunal_decision","classIds":["support"],"failureModeIds":["policy-conflict","identity-confusion","ungrounded-answer","stale-retrieval"],"lesson":"The decision says a customer used Air Canada's website chatbot while looking up a bereavement fare. The chatbot said a reduced rate could be requested within 90 days, including after travel. The linked bereavement page said the policy did not apply after travel was complete. An Air Canada representative later called the chatbot's words misleading. The tribunal found negligent misrepresentation, rejected the argument that the chatbot was a separate legal entity, and ordered Air Canada to pay $812.02. A support agent must not choose between two of the operator's pages, and the operator owns the answer.","sourceUrl":"https://decisions.civilresolutionbc.ca/crt/crtd/en/525448/1/document.do","asOf":"2026-10-03","notes":"Fetched the tribunal document (PDF) dated February 14, 2024, Tribunal Member Christopher C. Rivers. The order is $650.88 damages, $36.14 pre-judgment interest, and $125 in CRT fees. CanLII's HTML page returned a block and was not used."}]}