Kit · 2026-10-03
Security review agent
security-review
Reads a change or a trace for secrets, overly broad permissions, and untrusted text copied into a tool prompt. It does not produce an attack.
Needs below are the JTBD ids already listed on this class, capped at 6. The pick is the current Depot Pick. The cheapest line is that job's matching sourced SKU, or unavailable. A free row is listed only when its jtbd ids include that job. This page does not rescore. cost=free does not hide that pick and does not turn a cheap paid SKU into a free one.
GET /api/kit?class=security-review&cost=free · ?cost=free · all kits · class JSON
Depot Pick: lakera · score as of 2026-09-14 · /api/jtbds/prompt-injection-firewall
Depot Pick for prompt-injection-firewall: lakera (asOf 2026-09-14). See GET /api/jtbds/prompt-injection-firewall + GET /api/methodology.
pickIsNotPriceWinner: false
Cheapest matching SKU: unavailable — Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null.
Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null.
Free-only recommendation
mode: free_only · paid pick stays lakera. Not a free resource. A cheap paid SKU is not in this list.
- LLM Guard · llm-guard · dev_tool · oss
Free resources
freeMatch: jtbd
- LLM Guard · llm-guard · dev_tool · oss
Depot Pick: nvidia-nemo-guardrails · score as of 2026-09-14 · /api/jtbds/llm-input-guardrails
Depot Pick for llm-input-guardrails: nvidia-nemo-guardrails (asOf 2026-09-14). See GET /api/jtbds/llm-input-guardrails + GET /api/methodology.
pickIsNotPriceWinner: false
Cheapest matching SKU: openai-moderation · omni-moderation-latest (moderation endpoint) · 0 usd_per_1k_requests · not a per-million-token rate. min sourced per-request classifier/moderation SKU among candidates (an officially free endpoint is a $0 row with its source); chat-token prices are not borrowed; not quality-adjusted
Only 1 candidate has a matching USD per 1k requests; nothing was compared, so pickIsPriceWinner is null.
Free-only recommendation
mode: free_only · paid pick stays nvidia-nemo-guardrails. Not a free resource. A cheap paid SKU is not in this list.
no_free_resource_for_job
Free resources
freeMatch: none
none
Depot Pick: arthur-ai · score as of 2026-09-14 · /api/jtbds/runtime-llm-firewall
Depot Pick for runtime-llm-firewall: arthur-ai (asOf 2026-09-14). See GET /api/jtbds/runtime-llm-firewall + GET /api/methodology.
pickIsNotPriceWinner: false
Cheapest matching SKU: unavailable — Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null.
Nothing was compared. guardrail/firewall/gateway job: priced per request, plan or contract, not per chat token; chat input $/1M is not borrowed and no comparable sourced per-request unit is stored pickIsPriceWinner is null.
Free-only recommendation
mode: free_only · paid pick stays arthur-ai. Not a free resource. A cheap paid SKU is not in this list.
no_free_resource_for_job
Free resources
freeMatch: none
none
Depot Pick: promptfoo · score as of 2026-10-19 · /api/jtbds/adversarial-red-team-evals
Depot Pick for adversarial-red-team-evals: promptfoo (asOf 2026-10-19). See GET /api/jtbds/adversarial-red-team-evals + GET /api/methodology.
pickIsNotPriceWinner: false
Cheapest matching SKU: unavailable — Nothing was compared. no comparable sourced unit for this job; chat input $/1M and other units were not borrowed pickIsPriceWinner is null.
Nothing was compared. no comparable sourced unit for this job; chat input $/1M and other units were not borrowed pickIsPriceWinner is null.
Free-only recommendation
mode: free_only · paid pick stays promptfoo. Not a free resource. A cheap paid SKU is not in this list.
- Promptfoo · promptfoo · dev_tool · oss
Free resources
freeMatch: jtbd
- Promptfoo · promptfoo · dev_tool · oss
Depot Pick: infisical · score as of 2026-09-14 · /api/jtbds/agent-secrets
Depot Pick for agent-secrets: infisical (asOf 2026-09-14). See GET /api/jtbds/agent-secrets + GET /api/methodology.
pickIsNotPriceWinner: false
Cheapest matching SKU: unavailable — Nothing was compared. mixed_secret_units: usd_per_identity_month and usd_per_user_month are both stored. They were not ranked together and were not blended. Cheapest stays null. pickIsPriceWinner is null.
Nothing was compared. mixed_secret_units: usd_per_identity_month and usd_per_user_month are both stored. They were not ranked together and were not blended. Cheapest stays null. pickIsPriceWinner is null.
Free-only recommendation
mode: free_only · paid pick stays infisical. Not a free resource. A cheap paid SKU is not in this list.
no_free_resource_for_job
Free resources
freeMatch: none
none
Depot Pick: doppler · score as of 2026-09-14 · /api/jtbds/secret-rotation-automation
Depot Pick for secret-rotation-automation: doppler (asOf 2026-09-14). See GET /api/jtbds/secret-rotation-automation + GET /api/methodology.
pickIsNotPriceWinner: false
Cheapest matching SKU: unavailable — Nothing was compared. mixed_secret_units: usd_per_identity_month and usd_per_user_month are both stored. They were not ranked together and were not blended. Cheapest stays null. pickIsPriceWinner is null.
Nothing was compared. mixed_secret_units: usd_per_identity_month and usd_per_user_month are both stored. They were not ranked together and were not blended. Cheapest stays null. pickIsPriceWinner is null.
Free-only recommendation
mode: free_only · paid pick stays doppler. Not a free resource. A cheap paid SKU is not in this list.
no_free_resource_for_job
Free resources
freeMatch: none
none
Failure modes
Prompt injection prompt-injection
Compare each tool call to the original user task. If the call only makes sense given text that arrived from a fetched page, file, or message, stop and ask. Record which source the instruction came from. This catalog does not store payloads or attack steps.
Over-refusal over-refusal
Log refusals separately from successes. Review a sample against the written policy. Do not treat a refusal as a pass, and do not weaken a safety check to raise the pass rate.
Eval contamination eval-contamination
Keep gold labels out of the agent prompt. Store the harness version beside the score. Do not compare a number across a benchmark revision without saying the revision changed.
Weak verification weak-verification
Success requires an end-state check the agent does not grade for itself: tests, a database diff, or a separate judge that cannot see the gold label.