{"asOf":"2026-10-03","doesNotChangeDepotPicks":true,"doesNotChangePriceSort":true,"doesNotChangeFreeIndex":true,"note":"Classification and failure modes are mechanisms. Case studies are included only when the source URL was fetched on 2026-10-03. A capability ceiling is not a company incident. Vendor figures are labeled as that vendor's report.","count":1,"catalogCount":27,"refused":["Replit agent database story (July 2025): The Register URL returned HTTP 403 and no first-party engineering postmortem was fetched. Not indexed.","CanLII HTML for 2024 BCCRT 149 returned a client-side block. The Civil Resolution Tribunal document was fetched instead and is the indexed source.","Cursor agent best-practices post loaded and is a product guide, not a postmortem. Not indexed as an incident. The University of Chicago study it mentions was not fetched.","Cognition 'Introducing Devin' (published 2024-03-12) duplicates the SWE-bench technical report. The report is indexed; the launch post is not a second study.","Anthropic many-shot jailbreaking research page loaded. It is an attack writeup, so it is not indexed and no procedure is stored.","OWASP LLM01:2025 was fetched and is cited on the prompt-injection failure mode only. Attack examples from that page are not copied into a case study.","Calder Lab and similar unsourced production-metric blogs were not fetched and are not indexed.","Google Research abstract for Data Cascades in High-Stakes AI loaded on 2026-10-03 (the abstract says 92% prevalence). Not indexed for data-etl: it is an interview study of ML data practice, not a pipeline job failure. The indexed data-etl study is the dbt Cloud status page."],"studies":[{"id":"appworld-2024","title":"AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents","year":2024,"studyKind":"capability_ceiling","classIds":["coding","personal-assistant"],"failureModeIds":["collateral-side-effects","weak-verification"],"lesson":"The abstract describes 9 apps and 457 APIs, and 750 tasks whose checks allow more than one solution while also flagging unexpected changes. It says GPT-4o solved about 49% of normal tasks and about 30% of challenge tasks. A task that reaches the goal and also mutates something else is still a failed check in this design.","sourceUrl":"https://arxiv.org/abs/2407.18901","asOf":"2026-10-03","notes":"Fetched the arXiv abstract. Capability ceiling, not a company incident."}]}