Agent Outcome Verification, Countersign Protocol & Fail-Closed Promotion Gate
Why 'Done' is a hypothesis until an external deterministic oracle checks it — and how Agent Provider Depot enforces fail-closed epistemic honesty across all 206 JTBDs.
Clerk's 3 Epistemic Assay Stamps ([⊙_⊙])
Requires public verified outcome count n > 0 AND bound Countersign receipts passing scripts/promotion-predicate.mjs.
Default honest label when pricing/sources are verified but public outcome count is n = 0. Cite line says 'editorial prior'.
Applied automatically when editorial review exceeds 30 days without re-verification or fails the promotion predicate.
Fail-Closed Promotion Gate: Why We Demoted Our Own Unearned Badges
In October 2026, independent auditor verification caught 11 of 20 frozen-cohort jobs (including postgres-vector) wearing verified_outcome_backed while public verified outcomes were n=0. We built scripts/promotion-predicate.mjs, wired it into npm run check:picks, and demoted all unearned labels to deterministic_source_reverified ('editorial prior') without mutating a single score or pick.
Countersign Outcome Verification Protocol
https://countersign.signalry.aiCountersign is built by Signalry (sibling to Agent Provider Depot). Because of that relationship, Agent Depot runs behind an explicit Neutrality Firewall: scripts/promotion-predicate.mjs fails closed on any unbound or unverified receipt.
cs claim "Deployed v0.15.0 to production" --check http_get:https://getagentdepot.com/api/health --check json_path:https://getagentdepot.com/api/health#$.status=ok
curl -X POST https://getagentdepot.com/api/outcomes/verify -H "Content-Type: application/json" -d '{"claimId":"cs-0dccf4b7","jtbdId":"coding-agent","providerId":"anthropic","decisionStatus":"accepted"}'Core Verification & Guardrail JTBDs
Wrap agent generation steps in a bounded runtime verifier loop (compiler/typecheck, unit tests, JSON schema validator, or deterministic SQL/DOM check) so the agent repairs its own errors before returning an answer — without entering infinite retry spirals.
Grade an agent's multi-step execution trajectory — tool selection, JSON argument schemas, call ordering, state mutations, and retry recovery — instead of only scoring the final text string.
Calibrate LLM-as-judge graders against human gold labels and deterministic assertions so your eval harness does not reward verbose, sycophantic, or self-preferring outputs.
Continuously generate adversarial attacks (indirect prompt injection in untrusted tool outputs, PII exfiltration, unauthorized tool calls, jailbreaks) against your agent, patch system guardrails, and verify that attack success rate drops without spiking false-positive refusals.
Execute untrusted LLM-generated code safely.