Agent Learning, Memory Consolidation & Self-Improvement
How autonomous agents evaluate their own traces, consolidate episodic/semantic memory, optimize prompts and skills in sandboxes, and gate every self-modification with deterministic verifiers.
The 6-Stage Closed-Loop Self-Improvement Cycle
GET /api/learning → sixStageLoopDeterministic Trace & Outcome Capture
Record tool calls, arguments, latencies, token spend, and terminal outcomes with OpenTelemetry spans so every failure is reproducible.
External Verifier & Promotion Predicate Check
Grade trajectories with an external oracle (compiler, unit test, schema check, or Countersign receipt) rather than uncalibrated LLM self-praise.
Failure Trace Clustering & Root-Cause Critique
Cluster failed trajectories by root cause (tool schema drift, context rot, stale SKU, prompt injection, or overclaimed confidence).
Rule, Skill, Prompt, or Adapter Synthesis
Synthesize the minimal durable fix: a guardrail rule, a unit-tested sandbox skill (SKILL.md), a DSPy-compiled prompt, or a DPO/LoRA adapter.
Held-Out Regression & Adversarial Replay Gate
Run the candidate fix against held-out golden tasks and adversarial red-team probes in CI. Reject any fix with regressionCount > 0.
Skill Library Promotion & Countersign Receipt
Commit the passing skill/gate into version control, log the cycle in data/self-improvement-log.json, and anchor the release with a Countersign receipt.
The 8 Agent Learning & Self-Improvement JTBDs
8 jobs · 305-provider frozen catalog · editorial prior (±0.5)Agent Episodic & Semantic Memory Consolidation (Cross-Session Learning)
Extract durable user preferences, domain facts, and past episode outcomes from raw conversation and tool traces so an agent learns across sessions without bloating the context window or poisoning future runs with stale hallucinations.
When not top pick: Need temporal bi-temporal knowledge-graph edges with explicit validity windows (`valid_from` / `invalid_at`) for changing facts — Zep (Graphiti); need explicit OS-style self-editing core/archival memory blocks managed by the agent via tool calls — Letta; need DIY SQL + vector episodic tables inside existing Postgres — Supabase pgvector.
Automated Prompt, Few-Shot & Agent Program Optimization (DSPy / MIPROv2 / TextGrad)
Compile and optimize system instructions, tool descriptions, and multi-hop few-shot demonstrations against a deterministic metric and golden dataset — replacing manual prompt trial-and-error with versioned, eval-gated prompt compilation.
When not top pick: Local CLI / YAML matrix optimization and adversarial red-teaming in CI with zero SaaS dependency — Promptfoo; LangGraph node-level prompt versioning and hub playgrounds — LangSmith; open-source MIT prompt management + dataset experiment linkage — Langfuse.
Failure Trace Clustering, Reflexion & Guardrail Rule Synthesis
Automatically cluster production agent failure spans (tool schema errors, retry loops, judge rejections, user thumbs-down), run root-cause reflection, and synthesize candidate routing rules or negative constraints into aregression dataset.
When not top pick: Open-source MIT self-hosted trace clustering, cost attribution, and dataset promotion — Langfuse; CI-first eval diffing and scorer feedback loops — Braintrust; OpenTelemetry-native span clustering and drift analysis — Arize Phoenix.
Executable Skill & Tool Library Crystallization (Voyager-Style Verified Code Reuse)
Turn multi-step trajectories that succeeded in an isolated sandbox into parameterized, unit-tested, reusable skills (`SKILL.md` + deterministic scripts) so future agent runs invoke a verified tool instead of re-deriving brittle multi-turn plans.
When not top pick: Need stateful, persistent agent memory blocks attached to an agent runtime — Letta; need the evaluation harness to grade candidate skills across 100+ test inputs before promotion — Braintrust; need full git-workspace container persistence across days — Daytona.
Trajectory SFT, DPO & GRPO Distillation (Teacher-to-Student Agent Fine-Tuning)
Distill verified high-scoring trajectories from an expensive frontier teacher model (plus rejected failure pairs) into a smaller, faster open-weight student model via LoRA SFT, Direct Preference Optimization (DPO), or Verifier-Guided GRPO.
When not top pick: Ultra-low-latency serverless LoRA multi-tenant deployment after tuning — Fireworks AI; experiment tracking, dataset lineage, and reward-curve telemetry across RL/DPO sweeps — Weights & Biases; raw multi-node H100/B200 clusters for custom Megatron/vLLM RL loops — Lambda or Modal.
Online Bandit & Pareto Model Routing Adaptation (Cost / Latency / Quality Learning)
Continuously adapt model and provider routing weights per task class using live latency, error-rate, and verifier pass-rate telemetry — routing routine steps to fast/cheap models while escalating hard steps to frontier reasoning models.
When not top pick: Enterprise AI gateway with semantic caching, guardrails, and virtual keys across teams — Portkey; observability-first proxy with header-based caching and cost analytics — Helicone; offline dataset Pareto scoring to set initial routing priors — Braintrust.
Continuous Adversarial Red-Teaming, Prompt-Injection & Policy Self-Hardening
Continuously generate adversarial attacks (indirect prompt injection in untrusted tool outputs, PII exfiltration, unauthorized tool calls, jailbreaks) against your agent, patch system guardrails, and verify that attack success rate drops without spiking false-positive refusals.
When not top pick: Low-latency inline runtime prompt-injection firewall in production — Lakera Guard; programmable input/output validator rails in Python/TS — Guardrails AI; managed enterprise hallucination and policy evaluation suites — Patronus AI.
Verifier-Backed Runtime Self-Correction & Critic Loops (Compile / Test / Schema Gate)
Wrap agent generation steps in a bounded runtime verifier loop (compiler/typecheck, unit tests, JSON schema validator, or deterministic SQL/DOM check) so the agent repairs its own errors before returning an answer — without entering infinite retry spirals.
When not top pick: Need deterministic Pydantic/JSON-schema output validation and re-asking without full code execution — Guardrails AI; need offline + CI evaluation of whether self-correction steps actually improve pass@1 vs pass@k cost — Braintrust; need headless DOM/network verification for web actions — Browserbase.
Compact Symptom-to-Prescription Endpoint (< 2 KB)
GET /api/learning/diagnose?symptom=repeating-tool-errorsPass any runtime learning failure symptom (memory-contradiction, prompt-drift, repeating-tool-errors, high-cost-per-task, or prompt-injection) for a single-screen JSON prescription:
{
"schemaVersion": "agentdepot.learning_diagnosis.v1",
"asOf": "2026-10-28",
"querySymptom": "repeating-tool-errors",
"matchedCategory": "missing_skill_crystallization",
"rootCauseDiagnosis": "Agent is re-deriving 10-step plans from scratch on every turn instead of invoking a tested helper script or SKILL.md.",
"immediateAction": "Extract the working trajectory into a parameterized script, test it inside an E2B sandbox, and save it as a reusable SKILL.md.",
"antiPatternToAvoid": "Never promote a synthesized skill into the library unless its unit tests pass in a clean sandbox.",
"recommendedPrimaryJob": {
"jtbdId": "skill-library-crystallization",
"depotPickProviderId": "e2b",
"depotPickName": "E2B",
"evidenceBasis": "editorial prior (deterministic_source_reverified)",
"decideUrl": "https://getagentdepot.com/api/decide?jtbd=skill-library-crystallization"
},
"recommendedSecondaryJob": {
"jtbdId": "verifier-backed-self-correction-loops",
"decideUrl": "https://getagentdepot.com/api/decide?jtbd=verifier-backed-self-correction-loops"
},
"recommendedFramework": "Voyager Skill Pattern + E2B microVM sandbox verification",
"playbookUrl": "https://getagentdepot.com/playbooks/agent-self-improvement.md"
}Reference Frameworks, Benchmarks & Open-Source Harnesses
When manual prompt tweaking plateaus and you have >= 30 labeled examples + a deterministic metric to compile instructions and few-shot demos.
Mechanism: Joint Bayesian optimization over instructions and bootstrapped few-shot demonstrations (MIPROv2).
When optimizing compound multi-step agent pipelines or code/molecule/prompt artifacts using structured textual gradients from a critic.
Mechanism: Backpropagates natural-language critiques through computation graphs of LLM calls.
When an agent fails a coding, reasoning, or tool-use episode and has a concrete environment error signal to reflect on before retrying.
Mechanism: Converts binary/scalar environment feedback into a structured verbal critique stored in episodic memory.
When your agent repeatedly solves similar multi-step tasks and should crystallize verified code into a reusable tool/skill library.
Mechanism: Iterative code synthesis inside a sandbox -> automated verification -> indexed skill library retrieval.
When you have 1,000+ verifier-filtered trajectories from a frontier teacher model and want to distill them into a fast open-weight student.
Mechanism: Filters rollouts via deterministic verifiers (tests/math/schemas) and optimizes policy weights via DPO or Group Relative Policy Optimization.
When benchmarking coding, terminal, and tool-use agents before and after a self-improvement change to prove non-regression.
Mechanism: Sandboxed execution harnesses paired with reproducible scorers on SWE-bench Verified, Terminal-Bench, and GAIA.
When testing whether your episodic/semantic memory layer (Mem0, Zep, Letta, pgvector) retrieves updated facts without temporal confusion.
Mechanism: Evaluates knowledge updates, temporal reasoning, multi-session synthesis, and abstention over long interaction histories.
When an agent finishes a deploy, audit, or self-improvement cycle and needs an independent verification receipt (cs-xxxx) rather than self-attestation.
Mechanism: Runs deterministic HTTP/DNS/TLS/JSON/git checks and issues signed receipts with counts_as_verified gating.
Live Proof: Agent Provider Depot's Own Self-Improvement Ledger
data/self-improvement-log.json · npm run check:self-improveAgent Provider Depot runs this exact 6-stage loop on itself. Below are our live self-improvement cycles (including the qint_cc0b640320dea3e6 fail-closed promotion gate remediation).
Observed defect: 11 of 20 frozen-cohort jobs (including postgres-vector -> supabase-pgvector) and 36 governance pickAudits wore promotionClass=verified_outcome_backed / high_verified (±0.35) while public verified outcome count n=0 and cited cs-xxxx receipts were unbound repo/deploy claims.
Root cause & gate synthesized: Promotion gate checked only whether countersignReceiptIds was non-empty in governance.json, without verifying that public outcome count n > 0 in data/insights.json or that cited cs-xxxx claims explicitly bound the providerId and jtbdId. → Enforced by scripts/promotion-predicate.mjs.
Observed defect: Mid-word ellipsis truncation on /api/decide pricingSummary, trailing comma in citation, raw underscore in UI scoreKind badge, and missing pricing unit match on new eval JTBDs.
Root cause & gate synthesized: Hard 280-char slice cut words mid-token instead of snapping to a word boundary, and new JTBDs needed explicit SKU unit coverage checks. → Enforced by scripts/check-limits.mjs + scripts/check-data-correctness.mjs.
Observed defect: Agents asking how to consolidate episodic memory, compile prompts with DSPy/MIPROv2, cluster failure traces, crystallize skills in sandboxes, distill traces via DPO/GRPO, or run verifier-backed self-correction loops had no dedicated JTBDs.
Root cause & gate synthesized: Catalog covered observability and evals (16 JTBDs) but lacked the closed-loop post-eval learning and self-improvement jobs (8 JTBDs). → Enforced by scripts/self-improve-audit.mjs.
Observed defect: volume=100k silently fell back to 1,000 tasks ($1.03) and returned autoActOk: true under maxMonthly=10; compact next links dropped volume/maxMonthly; /api/resolve had gpt-5.4 vs gpt-6.1-sol mismatch and lacked advisoryDefaultOnly on unsupported constraints; /benchmark/runs/run_verified_depot_router_v0130 threw 500; /benchmark/leaderboard mixed officialEligible:false smoke runs into Authority Standing; and /api/blueprint?archetype=coding-agent-stack&budget=0 replaced the archetype instead of substituting $0 slots.
Root cause & gate synthesized: Number(raw) in parseContext rejected shorthand suffixes (100k, 2.5k, 1m) and fell back to null (triggering the 1,000-task starter quote in actionableForJob), while continuation URLs, blueprint budget=0 substitution, and benchmark run detail templates lacked strict invariant checks. → Enforced by scripts/self-improve-audit.mjs.