{"asOf":"2026-10-28","storesIpAddresses":false,"archetype":"eval-and-ci-pipeline","requestedArchetype":"eval-and-ci-pipeline","budgetAppliedUsd":null,"title":"4-Stage Agent Evaluation, Red-Team & CI Regression Stack","summary":"Complete 'Test Your Agent' harness combining public benchmarks, isolated sandboxes, pass^3 regression gates, and adversarial security probes.","estimatedMonthlyUsd":{"hobby":0,"startup":80,"scale":650},"oneLineInstallCmd":"npm install braintrust langfuse @e2b/code-interpreter promptfoo","requiredEnvVars":["BRAINTRUST_API_KEY","LANGFUSE_PUBLIC_KEY","LANGFUSE_SECRET_KEY","E2B_API_KEY","OPENAI_API_KEY"],"wiringNotes":"Run Promptfoo + Braintrust in GitHub Actions on every PR; stream production spans to Langfuse; execute untrusted eval code in E2B.","slots":[{"role":"CI Eval & Dataset Gate","jtbd":"prompt-eval-ci","providerId":"braintrust","providerName":"Braintrust","editorialOverall":8.23,"activeSku":"braintrust-pro","envVars":["BRAINTRUST_API_KEY"],"smokeTestCmd":"curl -sS https://api.braintrust.dev/v1/project -H \"Authorization: Bearer $BRAINTRUST_API_KEY\"","day2Gotcha":"Broader tracing competitors exist","whyItFits":"Blocks PRs when pass^3 consistency or costPerSolvedTaskUsd regresses.","decideUrl":"/api/decide?jtbd=prompt-eval-ci"},{"role":"OTel Production Tracing","jtbd":"llm-observability","providerId":"langfuse","providerName":"Langfuse","editorialOverall":8.66,"activeSku":"langfuse-cloud-core","envVars":["LANGFUSE_PUBLIC_KEY","LANGFUSE_SECRET_KEY"],"smokeTestCmd":"curl -sS https://cloud.langfuse.com/api/public/health","day2Gotcha":"Always await langfuse.flushAsync() in serverless waitUntil() or finally{} blocks.","whyItFits":"Captures production failures and converts them into golden eval datasets.","decideUrl":"/api/decide?jtbd=llm-observability"},{"role":"Isolated Benchmark Sandbox","jtbd":"code-sandbox","providerId":"e2b","providerName":"E2B","editorialOverall":8.56,"activeSku":"e2b-sandbox-1vcpu-per-s","envVars":["E2B_API_KEY"],"smokeTestCmd":"curl -sS https://api.e2b.dev/sandboxes -H \"X-API-Key: $E2B_API_KEY\"","day2Gotcha":"Explicitly call sandbox.kill() in a finally{} block so idle microVMs do not accrue vCPU-second charges.","whyItFits":"Runs SWE-bench Verified and Terminal-Bench tasks in clean microVMs.","decideUrl":"/api/decide?jtbd=agent-sandbox"},{"role":"Adversarial Red-Team & Prompt Injection","jtbd":"adversarial-safety-redteaming-evals","providerId":"promptfoo","providerName":"Promptfoo","editorialOverall":8.42,"activeSku":"promptfoo-oss","envVars":["OPENAI_API_KEY"],"smokeTestCmd":"test -n \"$OPENAI_API_KEY\" && echo \"OPENAI_API_KEY configured\"","day2Gotcha":"Less polished product UI than Braintrust","whyItFits":"Automated OWASP LLM Top-10 and jailbreak probes in GitHub Actions.","decideUrl":"/api/decide?jtbd=adversarial-safety-redteaming-evals"}],"cite":"Agent Provider Depot Blueprint (eval-and-ci-pipeline, as of 2026-10-28). https://getagentdepot.com/blueprints"}