{"asOf":"2026-10-03","doesNotChangeDepotPicks":true,"doesNotChangePriceSort":true,"doesNotRescore":true,"caps":{"needs":6,"freePerNeed":3,"caseStudies":3},"class":{"id":"security-review","name":"Security review agent","definition":"Reads a change or a trace for secrets, overly broad permissions, and untrusted text copied into a tool prompt. It does not produce an attack.","needs":{"jtbdIds":["prompt-injection-firewall","llm-input-guardrails","runtime-llm-firewall","adversarial-red-team-evals","agent-secrets","secret-rotation-automation","diff-aware-pr-review"],"freeKinds":["free_docs","dev_tool"],"capabilityTags":["safety"],"plain":["A findings list with file and line, not a payload","Secrets detected are redacted in the report"]},"failureModeIds":["prompt-injection","over-refusal","eval-contamination","weak-verification"],"exampleTasks":["List secrets and broad permissions in a diff","Flag a prompt that concatenates fetched page text into a tool instruction, without quoting an attack"]},"needsConsidered":["prompt-injection-firewall","llm-input-guardrails","runtime-llm-firewall","adversarial-red-team-evals","agent-secrets","secret-rotation-automation"],"needsOmitted":["diff-aware-pr-review"],"jobs":[{"jtbdId":"prompt-injection-firewall","decide":{"available":true,"unavailableReason":null,"providerId":"lakera","scoreAsOf":"2026-09-14","scoreAgeDays":43,"scoreStale":true,"runnerUpId":"nvidia-nemo-guardrails","cite":"Depot Pick for prompt-injection-firewall: lakera (asOf 2026-09-14). See GET /api/jtbds/prompt-injection-firewall + GET /api/methodology.","citePath":"/api/jtbds/prompt-injection-firewall"},"quote":{"available":true,"unavailableReason":null,"cheapest":null,"recommendedWorkloadQuote":null,"representativeQuote":null,"estimatedTaskCost":null,"reason":"Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null.","priceWinnerNote":"Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null."},"pickIsNotPriceWinner":false,"freeResources":[{"id":"llm-guard","name":"LLM Guard","kind":"dev_tool","cost":"oss","substitutable":true,"sourceUrl":"https://raw.githubusercontent.com/protectai/llm-guard/main/LICENSE"}],"freeMatch":"jtbd"},{"jtbdId":"llm-input-guardrails","decide":{"available":true,"unavailableReason":null,"providerId":"nvidia-nemo-guardrails","scoreAsOf":"2026-09-14","scoreAgeDays":43,"scoreStale":true,"runnerUpId":"lakera","cite":"Depot Pick for llm-input-guardrails: nvidia-nemo-guardrails (asOf 2026-09-14). See GET /api/jtbds/llm-input-guardrails + GET /api/methodology.","citePath":"/api/jtbds/llm-input-guardrails"},"quote":{"available":true,"unavailableReason":null,"cheapest":{"providerId":"openai-moderation","slug":"openai-moderation","name":"OpenAI Moderation","skuId":"openai-moderation-omni-free","sku":"omni-moderation-latest (moderation endpoint)","item":"omni-moderation-latest (moderation endpoint)","plan":"omni-moderation-latest (moderation endpoint) — Officially free: 'The moderation endpoint is free to use'","isFloorPrice":false,"floorWarning":null,"skuKind":"classifier_request","unit":"usd_per_1k_requests","amountUsd":0,"comparison":"min sourced per-request classifier/moderation SKU among candidates (an officially free endpoint is a $0 row with its source); chat-token prices are not borrowed; not quality-adjusted","usdPerMillionInputTokens":null,"sourceUrl":"https://developers.openai.com/api/docs/guides/moderation","asOf":"2026-10-21"},"recommendedWorkloadQuote":null,"representativeQuote":null,"estimatedTaskCost":null,"reason":null,"priceWinnerNote":"Only 1 candidate has a matching USD per 1k requests; nothing was compared, so pickIsPriceWinner is null."},"pickIsNotPriceWinner":false,"freeResources":[],"freeMatch":"none"},{"jtbdId":"runtime-llm-firewall","decide":{"available":true,"unavailableReason":null,"providerId":"arthur-ai","scoreAsOf":"2026-09-14","scoreAgeDays":43,"scoreStale":true,"runnerUpId":"lakera","cite":"Depot Pick for runtime-llm-firewall: arthur-ai (asOf 2026-09-14). See GET /api/jtbds/runtime-llm-firewall + GET /api/methodology.","citePath":"/api/jtbds/runtime-llm-firewall"},"quote":{"available":true,"unavailableReason":null,"cheapest":null,"recommendedWorkloadQuote":null,"representativeQuote":null,"estimatedTaskCost":null,"reason":"Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null.","priceWinnerNote":"Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null."},"pickIsNotPriceWinner":false,"freeResources":[],"freeMatch":"none"},{"jtbdId":"adversarial-red-team-evals","decide":{"available":true,"unavailableReason":null,"providerId":"promptfoo","scoreAsOf":"2026-10-19","scoreAgeDays":8,"scoreStale":false,"runnerUpId":"patronus","cite":"Depot Pick for adversarial-red-team-evals: promptfoo (asOf 2026-10-19). See GET /api/jtbds/adversarial-red-team-evals + GET /api/methodology.","citePath":"/api/jtbds/adversarial-red-team-evals"},"quote":{"available":true,"unavailableReason":null,"cheapest":null,"recommendedWorkloadQuote":null,"representativeQuote":null,"estimatedTaskCost":null,"reason":"Nothing was compared. no comparable sourced unit for this job; chat input $/1M and other units were not borrowed pickIsPriceWinner is null.","priceWinnerNote":"Nothing was compared. no comparable sourced unit for this job; chat input $/1M and other units were not borrowed pickIsPriceWinner is null."},"pickIsNotPriceWinner":false,"freeResources":[{"id":"promptfoo","name":"Promptfoo","kind":"dev_tool","cost":"oss","substitutable":true,"sourceUrl":"https://github.com/promptfoo/promptfoo/blob/main/LICENSE"}],"freeMatch":"jtbd"},{"jtbdId":"agent-secrets","decide":{"available":true,"unavailableReason":null,"providerId":"infisical","scoreAsOf":"2026-09-14","scoreAgeDays":43,"scoreStale":true,"runnerUpId":"doppler","cite":"Depot Pick for agent-secrets: infisical (asOf 2026-09-14). See GET /api/jtbds/agent-secrets + GET /api/methodology.","citePath":"/api/jtbds/agent-secrets"},"quote":{"available":true,"unavailableReason":null,"cheapest":null,"recommendedWorkloadQuote":{"providerId":"infisical","providerName":"Infisical","skuId":"infisical-secrets-pro-annual","item":"Secrets Management Pro annual","plan":"Secrets Management Pro annual — Per identity per month when billed annually","unit":"usd_per_identity_month","amountUsd":20,"outputSkuId":null,"outputItem":null,"outputAmountUsd":null,"outputUnit":null,"sourceUrl":"https://infisical.com/pricing","asOf":"2026-10-03","pricingLastVerifiedAt":"2026-10-03","isFlagshipForJob":false,"differsFromFloorCheapest":false,"representativeQuote":{"skuId":"infisical-secrets-pro-annual","item":"Secrets Management Pro annual","plan":"Secrets Management Pro annual — Per identity per month when billed annually","amountUsd":20,"inputOrUnitUsd":20,"unit":"usd_per_identity_month","outputUsd":null,"outputUnit":null,"sourceUrl":"https://infisical.com/pricing","asOf":"2026-10-03"},"estimatedTaskCost":{"tasks":1000,"retryAndToolOverheadMultiplier":1.15,"retryToolOverheadMultiplier":1.15,"unitsPerTask":1,"baseCostPer1kTasksUsd":20,"totalUsd":23,"estimatedTotalUsd":23,"perTaskUsd":0.023,"estimatedCostPerTaskUsd":0.023,"formula":"Monthly base plan $20/mo covering 1,000 tasks (1.15x overhead buffer)","assumptions":"Monthly subscription plan amortized over 1,000 tasks/month."}},"representativeQuote":{"skuId":"infisical-secrets-pro-annual","item":"Secrets Management Pro annual","plan":"Secrets Management Pro annual — Per identity per month when billed annually","amountUsd":20,"inputOrUnitUsd":20,"unit":"usd_per_identity_month","outputUsd":null,"outputUnit":null,"sourceUrl":"https://infisical.com/pricing","asOf":"2026-10-03"},"estimatedTaskCost":{"tasks":1000,"retryAndToolOverheadMultiplier":1.15,"retryToolOverheadMultiplier":1.15,"unitsPerTask":1,"baseCostPer1kTasksUsd":20,"totalUsd":23,"estimatedTotalUsd":23,"perTaskUsd":0.023,"estimatedCostPerTaskUsd":0.023,"formula":"Monthly base plan $20/mo covering 1,000 tasks (1.15x overhead buffer)","assumptions":"Monthly subscription plan amortized over 1,000 tasks/month."},"reason":"Nothing was compared. mixed_secret_units: usd_per_identity_month and usd_per_user_month are both stored. They were not ranked together and were not blended. Cheapest stays null. pickIsPriceWinner is null.","priceWinnerNote":"Nothing was compared. mixed_secret_units: usd_per_identity_month and usd_per_user_month are both stored. They were not ranked together and were not blended. Cheapest stays null. pickIsPriceWinner is null."},"pickIsNotPriceWinner":false,"freeResources":[],"freeMatch":"none"},{"jtbdId":"secret-rotation-automation","decide":{"available":true,"unavailableReason":null,"providerId":"doppler","scoreAsOf":"2026-09-14","scoreAgeDays":43,"scoreStale":true,"runnerUpId":"infisical","cite":"Depot Pick for secret-rotation-automation: doppler (asOf 2026-09-14). See GET /api/jtbds/secret-rotation-automation + GET /api/methodology.","citePath":"/api/jtbds/secret-rotation-automation"},"quote":{"available":true,"unavailableReason":null,"cheapest":null,"recommendedWorkloadQuote":{"providerId":"doppler","providerName":"Doppler","skuId":"doppler-developer-additional-user","item":"Developer additional user","plan":"Developer additional user — Per additional user per month after 3 included users","unit":"usd_per_user_month","amountUsd":8,"outputSkuId":null,"outputItem":null,"outputAmountUsd":null,"outputUnit":null,"sourceUrl":"https://www.doppler.com/pricing","asOf":"2026-10-03","pricingLastVerifiedAt":"2026-10-03","isFlagshipForJob":false,"differsFromFloorCheapest":false,"representativeQuote":{"skuId":"doppler-developer-additional-user","item":"Developer additional user","plan":"Developer additional user — Per additional user per month after 3 included users","amountUsd":8,"inputOrUnitUsd":8,"unit":"usd_per_user_month","outputUsd":null,"outputUnit":null,"sourceUrl":"https://www.doppler.com/pricing","asOf":"2026-10-03"},"estimatedTaskCost":{"tasks":1000,"retryAndToolOverheadMultiplier":1.15,"retryToolOverheadMultiplier":1.15,"unitsPerTask":1,"baseCostPer1kTasksUsd":8,"totalUsd":9.2,"estimatedTotalUsd":9.2,"perTaskUsd":0.0092,"estimatedCostPerTaskUsd":0.0092,"formula":"Monthly base plan $8/mo covering 1,000 tasks (1.15x overhead buffer)","assumptions":"Monthly subscription plan amortized over 1,000 tasks/month."}},"representativeQuote":{"skuId":"doppler-developer-additional-user","item":"Developer additional user","plan":"Developer additional user — Per additional user per month after 3 included users","amountUsd":8,"inputOrUnitUsd":8,"unit":"usd_per_user_month","outputUsd":null,"outputUnit":null,"sourceUrl":"https://www.doppler.com/pricing","asOf":"2026-10-03"},"estimatedTaskCost":{"tasks":1000,"retryAndToolOverheadMultiplier":1.15,"retryToolOverheadMultiplier":1.15,"unitsPerTask":1,"baseCostPer1kTasksUsd":8,"totalUsd":9.2,"estimatedTotalUsd":9.2,"perTaskUsd":0.0092,"estimatedCostPerTaskUsd":0.0092,"formula":"Monthly base plan $8/mo covering 1,000 tasks (1.15x overhead buffer)","assumptions":"Monthly subscription plan amortized over 1,000 tasks/month."},"reason":"Nothing was compared. mixed_secret_units: usd_per_identity_month and usd_per_user_month are both stored. They were not ranked together and were not blended. Cheapest stays null. pickIsPriceWinner is null.","priceWinnerNote":"Nothing was compared. mixed_secret_units: usd_per_identity_month and usd_per_user_month are both stored. They were not ranked together and were not blended. Cheapest stays null. pickIsPriceWinner is null."},"pickIsNotPriceWinner":false,"freeResources":[],"freeMatch":"none"}],"failures":[{"id":"prompt-injection","name":"Prompt injection","detection":"Compare each tool call to the original user task. If the call only makes sense given text that arrived from a fetched page, file, or message, stop and ask. Record which source the instruction came from. This catalog does not store payloads or attack steps."},{"id":"over-refusal","name":"Over-refusal","detection":"Log refusals separately from successes. Review a sample against the written policy. Do not treat a refusal as a pass, and do not weaken a safety check to raise the pass rate."},{"id":"eval-contamination","name":"Eval contamination","detection":"Keep gold labels out of the agent prompt. Store the harness version beside the score. Do not compare a number across a benchmark revision without saying the revision changed."},{"id":"weak-verification","name":"Weak verification","detection":"Success requires an end-state check the agent does not grade for itself: tests, a database diff, or a separate judge that cannot see the gold label."}],"failuresOmittedBecauseMissing":[],"caseStudies":[{"id":"anthropic-computer-use-2024","title":"Developing a computer use model","sourceUrl":"https://www.anthropic.com/news/developing-computer-use","year":2024,"studyKind":"vendor_writeup"},{"id":"willison-prompt-injection-2023","title":"Prompt injection: What's the worst that can happen?","sourceUrl":"https://simonwillison.net/2023/Apr/14/worst-that-can-happen/","year":2023,"studyKind":"mechanism_study"}],"caseStudiesRelated":2,"caseStudiesOmitted":0,"caseStudiesReason":null,"links":{"kit":"/api/kit?class=security-review","page":"/agents/kit/security-review","agentClass":"/api/agent-classes/security-review","failures":"/api/failures","caseStudies":"/api/case-studies?class=security-review"},"recommendation":{"mode":"default"},"stackContext":{"host":null,"framework":null,"repo":null,"lang":null,"existing":[],"volume":null,"sender":null,"detectedFromTask":[]},"contextAdvice":[],"firstActions":["https://getagentdepot.com/api/hub","https://getagentdepot.com/api/decide?jtbd=prompt-injection-firewall","https://getagentdepot.com/api/quote?jtbd=prompt-injection-firewall","https://getagentdepot.com/api/kit?class=security-review","https://getagentdepot.com/api/failures"],"honesty":["Picks are the current Depot Pick from GET /api/decide. This kit does not rescore.","Cheapest SKU is copied from GET /api/quote for that job. Input and output are not blended. A chat SKU is not borrowed for a non-chat job. If quote.cheapest is null, there is no matching sourced price.","pickIsNotPriceWinner is true only when that job's Depot Pick and cheapest matching SKU both exist and the quote says the pick is not the price winner. If either is missing, the field is false.","freeResources are existing /api/free items whose jtbdIds include that job, up to 3, sorted by id. If none list the job, freeResources is [] and freeMatch is none. Kind, provider, and docs are not fallbacks. Free does not change the pick.","firstActions are endpoint URLs for this class: hub, decide for the first listed need, quote for that need, this kit, and /api/failures. No other step is added.","Failures are the modes already listed on the class. Case studies are the first 3 related studies in catalog order, with the source URL already stored. Omitted studies stay on GET /api/case-studies?class=."]}