# Playbook: The Closed-Loop Self-Improving Agent (v0.15.0)

> **Machine JSON Twin:** `GET https://getagentdepot.com/api/playbooks/agent-self-improvement`
> **Interactive Diagnosis (< 2 KB):** `GET https://getagentdepot.com/api/learning/diagnose?symptom=repeating-tool-errors`
> **Learning Hub:** `https://getagentdepot.com/learning` · **JSON:** `GET https://getagentdepot.com/api/learning`

## First-Principles Rule: Never Run Ungrounded Self-Critique

Intrinsic LLM self-correction ("Are you sure? Critique your answer and try again") without an external ground-truth oracle degrades accuracy and multiplies token spend by 2–4x (Huang et al.). Every trustworthy self-improvement cycle MUST be anchored by an **external deterministic verifier**:
1. A compiler, typechecker, or unit test suite inside an isolated sandbox (`e2b`).
2. A JSON Schema / Pydantic validator or policy rail (`guardrails-ai`, `promptfoo`).
3. A held-out golden dataset scorer (`braintrust`, `langsmith`, `langfuse`).
4. An independent public-API outcome check and receipt (`countersign`, `scripts/promotion-predicate.mjs`).

---

## The 6-Stage Closed-Loop Self-Improvement Cycle

### 01 // CAPTURE & TRACE — Deterministic Trace & Outcome Capture
- **What to do:** Record tool calls, arguments, latencies, token spend, and terminal outcomes with OpenTelemetry spans so every failure is reproducible.
- **Primary JTBD:** `agent-trajectory-tool-use-evals` (`GET https://getagentdepot.com/api/decide?jtbd=agent-trajectory-tool-use-evals`)
- **Hard invariant:** Never optimize what you cannot replay deterministically.

### 02 // GRADE & FALSIFY — External Verifier & Promotion Predicate Check
- **What to do:** Grade trajectories with an external oracle (compiler, unit test, schema check, or Countersign receipt) rather than uncalibrated LLM self-praise.
- **Primary JTBD:** `verifier-backed-self-correction-loops` (`GET https://getagentdepot.com/api/decide?jtbd=verifier-backed-self-correction-loops`)
- **Hard invariant:** Separate verified_outcome_backed from deterministic_source_reverified (editorial prior).

### 03 // REFLECT & CLUSTER — Failure Trace Clustering & Root-Cause Critique
- **What to do:** Cluster failed trajectories by root cause (tool schema drift, context rot, stale SKU, prompt injection, or overclaimed confidence).
- **Primary JTBD:** `failure-trace-reflection-rule-synthesis` (`GET https://getagentdepot.com/api/decide?jtbd=failure-trace-reflection-rule-synthesis`)
- **Hard invariant:** Name the exact violated invariant before drafting a fix.

### 04 // SYNTHESIZE FIX — Rule, Skill, Prompt, or Adapter Synthesis
- **What to do:** Synthesize the minimal durable fix: a guardrail rule, a unit-tested sandbox skill (SKILL.md), a DSPy-compiled prompt, or a DPO/LoRA adapter.
- **Primary JTBD:** `prompt-program-self-optimization` (`GET https://getagentdepot.com/api/decide?jtbd=prompt-program-self-optimization`)
- **Hard invariant:** Prefer a deterministic script gate over adding another paragraph of prompt instructions.

### 05 // REPLAY & GATE — Held-Out Regression & Adversarial Replay Gate
- **What to do:** Run the candidate fix against held-out golden tasks and adversarial red-team probes in CI. Reject any fix with regressionCount > 0.
- **Primary JTBD:** `automated-red-teaming-self-hardening` (`GET https://getagentdepot.com/api/decide?jtbd=automated-red-teaming-self-hardening`)
- **Hard invariant:** Require regressionCount === 0 on held-out tasks before promotion.

### 06 // CRYSTALLIZE & WITNESS — Skill Library Promotion & Countersign Receipt
- **What to do:** Commit the passing skill/gate into version control, log the cycle in data/self-improvement-log.json, and anchor the release with a Countersign receipt.
- **Primary JTBD:** `skill-library-crystallization` (`GET https://getagentdepot.com/api/decide?jtbd=skill-library-crystallization`)
- **Hard invariant:** File a verifiable receipt (cs claim) and log public corrections openly.

---

## The 8 Agent Learning & Self-Improvement JTBDs

1. **`agent-episodic-semantic-memory-learning`** — Agent Episodic & Semantic Memory Consolidation (Cross-Session Learning)
   - **Depot Pick (editorial prior):** `mem0` (Mem0, score 7.59/10 as of 2026-10-28)
   - **Runners-up:** `zep`, `letta`, `supabase-pgvector`
   - **When not to use pick:** Need temporal bi-temporal knowledge-graph edges with explicit validity windows (`valid_from` / `invalid_at`) for changing facts — Zep (Graphiti); need explicit OS-style self-editing core/archival memory blocks managed by the agent via tool calls — Letta; need DIY SQL + vector episodic tables inside existing Postgres — Supabase pgvector.
   - **Machine decide:** `GET https://getagentdepot.com/api/decide?jtbd=agent-episodic-semantic-memory-learning`

2. **`prompt-program-self-optimization`** — Automated Prompt, Few-Shot & Agent Program Optimization (DSPy / MIPROv2 / TextGrad)
   - **Depot Pick (editorial prior):** `braintrust` (Braintrust, score 8.23/10 as of 2026-10-28)
   - **Runners-up:** `promptfoo`, `langsmith`, `langfuse`
   - **When not to use pick:** Local CLI / YAML matrix optimization and adversarial red-teaming in CI with zero SaaS dependency — Promptfoo; LangGraph node-level prompt versioning and hub playgrounds — LangSmith; open-source MIT prompt management + dataset experiment linkage — Langfuse.
   - **Machine decide:** `GET https://getagentdepot.com/api/decide?jtbd=prompt-program-self-optimization`

3. **`failure-trace-reflection-rule-synthesis`** — Failure Trace Clustering, Reflexion & Guardrail Rule Synthesis
   - **Depot Pick (editorial prior):** `langsmith` (LangSmith, score 8.09/10 as of 2026-10-28)
   - **Runners-up:** `langfuse`, `braintrust`, `arize-phoenix`
   - **When not to use pick:** Open-source MIT self-hosted trace clustering, cost attribution, and dataset promotion — Langfuse; CI-first eval diffing and scorer feedback loops — Braintrust; OpenTelemetry-native span clustering and drift analysis — Arize Phoenix.
   - **Machine decide:** `GET https://getagentdepot.com/api/decide?jtbd=failure-trace-reflection-rule-synthesis`

4. **`skill-library-crystallization`** — Executable Skill & Tool Library Crystallization (Voyager-Style Verified Code Reuse)
   - **Depot Pick (editorial prior):** `e2b` (E2B, score 8.56/10 as of 2026-10-28)
   - **Runners-up:** `letta`, `braintrust`, `daytona`
   - **When not to use pick:** Need stateful, persistent agent memory blocks attached to an agent runtime — Letta; need the evaluation harness to grade candidate skills across 100+ test inputs before promotion — Braintrust; need full git-workspace container persistence across days — Daytona.
   - **Machine decide:** `GET https://getagentdepot.com/api/decide?jtbd=skill-library-crystallization`

5. **`preference-dpo-sft-trace-distillation`** — Trajectory SFT, DPO & GRPO Distillation (Teacher-to-Student Agent Fine-Tuning)
   - **Depot Pick (editorial prior):** `together` (Together AI, score 8.2/10 as of 2026-10-28)
   - **Runners-up:** `fireworks`, `weights-biases`, `modal`
   - **When not to use pick:** Ultra-low-latency serverless LoRA multi-tenant deployment after tuning — Fireworks AI; experiment tracking, dataset lineage, and reward-curve telemetry across RL/DPO sweeps — Weights & Biases; raw multi-node H100/B200 clusters for custom Megatron/vLLM RL loops — Lambda or Modal.
   - **Machine decide:** `GET https://getagentdepot.com/api/decide?jtbd=preference-dpo-sft-trace-distillation`

6. **`online-bandit-model-routing-adaptation`** — Online Bandit & Pareto Model Routing Adaptation (Cost / Latency / Quality Learning)
   - **Depot Pick (editorial prior):** `openrouter` (OpenRouter, score 8/10 as of 2026-10-28)
   - **Runners-up:** `portkey`, `helicone`, `braintrust`
   - **When not to use pick:** Enterprise AI gateway with semantic caching, guardrails, and virtual keys across teams — Portkey; observability-first proxy with header-based caching and cost analytics — Helicone; offline dataset Pareto scoring to set initial routing priors — Braintrust.
   - **Machine decide:** `GET https://getagentdepot.com/api/decide?jtbd=online-bandit-model-routing-adaptation`

7. **`automated-red-teaming-self-hardening`** — Continuous Adversarial Red-Teaming, Prompt-Injection & Policy Self-Hardening
   - **Depot Pick (editorial prior):** `promptfoo` (Promptfoo, score 8.42/10 as of 2026-10-28)
   - **Runners-up:** `lakera`, `guardrails-ai`, `patronus`
   - **When not to use pick:** Low-latency inline runtime prompt-injection firewall in production — Lakera Guard; programmable input/output validator rails in Python/TS — Guardrails AI; managed enterprise hallucination and policy evaluation suites — Patronus AI.
   - **Machine decide:** `GET https://getagentdepot.com/api/decide?jtbd=automated-red-teaming-self-hardening`

8. **`verifier-backed-self-correction-loops`** — Verifier-Backed Runtime Self-Correction & Critic Loops (Compile / Test / Schema Gate)
   - **Depot Pick (editorial prior):** `e2b` (E2B, score 8.56/10 as of 2026-10-28)
   - **Runners-up:** `guardrails-ai`, `braintrust`, `browserbase`
   - **When not to use pick:** Need deterministic Pydantic/JSON-schema output validation and re-asking without full code execution — Guardrails AI; need offline + CI evaluation of whether self-correction steps actually improve pass@1 vs pass@k cost — Braintrust; need headless DOM/network verification for web actions — Browserbase.
   - **Machine decide:** `GET https://getagentdepot.com/api/decide?jtbd=verifier-backed-self-correction-loops`

---

## Reference Frameworks, Benchmarks & Harnesses

- **DSPy & MIPROv2 (Stanford NLP)** (Prompt & Program Compilation) — When manual prompt tweaking plateaus and you have >= 30 labeled examples + a deterministic metric to compile instructions and few-shot demos. · Mechanism: Joint Bayesian optimization over instructions and bootstrapped few-shot demonstrations (MIPROv2). · Source: https://dspy.ai
- **TextGrad (Zou Lab / Stanford)** (Textual Backpropagation) — When optimizing compound multi-step agent pipelines or code/molecule/prompt artifacts using structured textual gradients from a critic. · Mechanism: Backpropagates natural-language critiques through computation graphs of LLM calls. · Source: https://github.com/zou-group/textgrad
- **Reflexion (Shinn et al.)** (Verbal Reinforcement & Episodic Critique) — When an agent fails a coding, reasoning, or tool-use episode and has a concrete environment error signal to reflect on before retrying. · Mechanism: Converts binary/scalar environment feedback into a structured verbal critique stored in episodic memory. · Source: https://arxiv.org/abs/2303.11366
- **Voyager Skill Library Pattern (Wang et al.)** (Executable Skill Crystallization) — When your agent repeatedly solves similar multi-step tasks and should crystallize verified code into a reusable tool/skill library. · Mechanism: Iterative code synthesis inside a sandbox -> automated verification -> indexed skill library retrieval. · Source: https://voyager.minedojo.org
- **STaR / ReST^EM / DPO / Verifier-Guided GRPO** (Trajectory Distillation & RL) — When you have 1,000+ verifier-filtered trajectories from a frontier teacher model and want to distill them into a fast open-weight student. · Mechanism: Filters rollouts via deterministic verifiers (tests/math/schemas) and optimizes policy weights via DPO or Group Relative Policy Optimization. · Source: https://arxiv.org/abs/2501.12948
- **SWE-agent / OpenHands / UK AISI Inspect AI** (Agent Evaluation & Sandboxed Harnesses) — When benchmarking coding, terminal, and tool-use agents before and after a self-improvement change to prove non-regression. · Mechanism: Sandboxed execution harnesses paired with reproducible scorers on SWE-bench Verified, Terminal-Bench, and GAIA. · Source: https://inspect.ai-safety-institute.org.uk
- **LongMemEval & LOCOMO** (Cross-Session Memory Benchmarks) — When testing whether your episodic/semantic memory layer (Mem0, Zep, Letta, pgvector) retrieves updated facts without temporal confusion. · Mechanism: Evaluates knowledge updates, temporal reasoning, multi-session synthesis, and abstention over long interaction histories. · Source: https://github.com/xiaowu0162/LongMemEval
- **Countersign Outcome Verification Protocol** (Cryptographic & Public-API Witnessing) — When an agent finishes a deploy, audit, or self-improvement cycle and needs an independent verification receipt (cs-xxxx) rather than self-attestation. · Mechanism: Runs deterministic HTTP/DNS/TLS/JSON/git checks and issues signed receipts with counts_as_verified gating. · Source: https://countersign.signalry.ai

---

## Live Case Study: How Agent Provider Depot Runs This Loop on Itself

In Cycle `SIL-2026-10-28-001` (quarantine receipt `qint_cc0b640320dea3e6`), an independent audit caught 11 of 20 frozen-cohort jobs (including `postgres-vector`) wearing `promotionClass: "verified_outcome_backed"` while public verified outcome count was `n = 0` and cited `cs-xxxx` receipts were unbound repo/deploy claims.

Instead of quietly editing a JSON field by hand, `directory-agent` executed the 6-stage loop:
1. **Captured** the exact discrepancy in `data/promotion-coherence-report.json` (`quarantineReceiptId: "qint_cc0b640320dea3e6"`).
2. **Graded** all 206 JTBDs and 20 frozen-cohort items with a new deterministic script (`scripts/promotion-predicate.mjs`).
3. **Reflected** on the root cause (checking non-empty receipt arrays without verifying `n > 0` public outcomes and provider/JTBD binding).
4. **Synthesized** `promotionDefects()` into `scripts/check-pick-authority.mjs` and demoted unearned labels to `deterministic_source_reverified` (`editorial_prior`, `±0.5`) while keeping `scoresMutated: false` and `picksMutated: false`.
5. **Replayed** 3 self-tests + 471 pick-authority invariants (`regressionCount: 0`).
6. **Crystallized** the gate into CI (`npm run check:picks` and `npm run check:self-improve`).