{"asOf":"2026-10-03","doesNotChangeDepotPicks":true,"doesNotChangePriceSort":true,"doesNotChangeFreeIndex":true,"note":"Classification and failure modes are mechanisms. Case studies are included only when the source URL was fetched on 2026-10-03. A capability ceiling is not a company incident. Vendor figures are labeled as that vendor's report.","id":"local-offline","name":"Local and offline agent","definition":"Runs without calling a hosted model API, using local weights, a local index, or an air-gapped serve path.","needs":{"jtbdIds":["air-gapped-model-serve","open-model-inference","self-host-vector","self-host-browser","wasm-edge-small-models","on-device-edge-batch","edge-sqlite-agent-db"],"freeKinds":["open_weights","local_runtime","free_vector","free_sandbox"],"capabilityTags":["oss","edge"],"plain":["No network call on the answer path","Say when the local corpus has no support"]},"failureModeIds":["silent-truncation","context-overflow","schema-drift","non-determinism","stale-retrieval","lost-in-the-middle"],"exampleTasks":["Answer from a local index and say when it has no passage","Refuse to call a hosted model when the task is marked offline"],"failures":[{"id":"silent-truncation","name":"Silent truncation","whatGoesWrong":"Tool output or a document is cut off without a marker, and the agent answers as if it saw the whole thing.","howToDetect":"Tools must return whether the payload was truncated and how long the original was. Fail the task if the answer depends on the missing span.","classIds":["rag","coding","data-etl","voice","local-offline","research"],"sourceUrls":["https://arxiv.org/abs/2310.08560"]},{"id":"context-overflow","name":"Context overflow","whatGoesWrong":"The document or the session does not fit the context window, and the agent answers from whatever fragment was left in view.","howToDetect":"If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.","classIds":["rag","personal-assistant","research","local-offline","coding"],"sourceUrls":["https://arxiv.org/abs/2310.08560"]},{"id":"schema-drift","name":"Schema drift","whatGoesWrong":"A tool, table, or file changed shape, and the agent still sends the old arguments or reads the old columns.","howToDetect":"Validate every tool call and every loaded table against the current schema. Quarantine mismatches instead of coercing them.","classIds":["coding","data-etl","local-offline","support"],"sourceUrls":[]},{"id":"non-determinism","name":"Non-determinism","whatGoesWrong":"The same task produces a different tool sequence or a different end state on another trial, so one success does not mean the agent is reliable.","howToDetect":"Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.","classIds":["support","coding","browser","scheduled-ops","eval-judge","voice"],"sourceUrls":["https://arxiv.org/abs/2406.12045"]},{"id":"stale-retrieval","name":"Stale retrieval","whatGoesWrong":"The answer is grounded in a passage, index, or policy page that is older than the source the operator now treats as canonical.","howToDetect":"Store the source URL and the time it was fetched. If a newer canonical page exists, or two stored sources disagree, do not answer from the older one.","classIds":["rag","research","support","data-etl","personal-assistant","sales-outreach","local-offline"],"sourceUrls":["https://decisions.civilresolutionbc.ca/crt/crtd/en/525448/1/document.do"]},{"id":"lost-in-the-middle","name":"Lost in the middle","whatGoesWrong":"The needed passage is in the prompt, but the model uses the beginning or the end and misses the middle of a long context.","howToDetect":"Ask the agent to name the passage it used. Spot-check questions whose only support sits in the middle of the context, not only at the edges.","classIds":["rag","research","local-offline"],"sourceUrls":["https://arxiv.org/abs/2307.03172"]}],"caseStudies":[{"id":"memgpt-2023","title":"MemGPT: Towards LLMs as Operating Systems","year":2023,"studyKind":"mechanism_study","classIds":["personal-assistant","rag","local-offline"],"failureModeIds":["context-overflow","silent-truncation"],"lesson":"The abstract says limited context windows handicap document analysis and multi-session chat, and describes moving text between tiers so the model sees a working set rather than the whole history. An assistant that answers a long document or a long memory from a single truncated window is the problem this paper is about. No accuracy number is stored because the abstract did not state one.","sourceUrl":"https://arxiv.org/abs/2310.08560","asOf":"2026-10-03","notes":"Fetched the arXiv abstract. A system paper, not an incident."}]}