Diff-aware pull-request review agents

Scores as of 2026-10-19 (8 days old).

Depot Pick

Anthropic · 8.54/10 for this job (curated estimate, high confidence)

Related to `coding-agent` but optimized for review posture (precision, rationale, patch safety). Confidence: high on model quality; product wrappers vary.

Runner-up

OpenAI · 8.33/10

Scores higher than the pick on: Latency / reliability, Developer & agent DX, Maturity / track record.

When not to use the pick

Bulk low-cost review of huge monorepos (DeepSeek/open hosts) with human confirm, or procurement locks you to Azure OpenAI.

Lock-in / migration

The pick has lower stored lock-in risk than runner-up OpenAI (6.5 vs 7.2). Self-host fallback listed for this job: reviewdog.

Sourced price (usd_per_million_input_tokens)

Pick's lowest sourced SKU: Haiku 5.5 (prompts up to 100K tokens) $0.1 (usd_per_million_input_tokens) — source, as of 2026-10-27, re-checked 2026-10-27

Cheapest matching SKU: Anthropic Haiku 5.5 (prompts up to 100K tokens) $0.1 — source, as of 2026-10-27, re-checked 2026-10-27. The pick is the cheapest.

Price history: first seen 2026-10-21 · last changed no change recorded · history JSON

Free option

• reviewdog (oss) — source, as of 2026-10-25

Failure modes to watch

• Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
• Eval contamination: Keep gold labels out of the agent prompt. Store the harness version beside the score. Do not compare a number across a benchmark revision without saying the revision changed.
• Weak verification: Success requires an end-state check the agent does not grade for itself: tests, a database diff, or a separate judge that cannot see the gold label.
• Benchmark protocol mismatch: Store assisted-or-not, subset, and harness version beside every cited score. Refuse a ranking that mixes those fields.
• Collateral side effects: Snapshot the workspace or app state, then diff it against an allowlist. A passing check plus an unexpected diff is a failure.
• Context overflow: If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.

Reported outcomes (submitted reports only)

This job: loading… JSON · reports never change scores · seed/test records excluded

Depot Pick (Anthropic) on this job: loading… JSON · reports never change scores · seed/test records excluded

Same-job, same-unit comparisons: Anthropic vs OpenAI · Anthropic vs Google AI (Gemini)

Cite: Agent Provider Depot, Depot Pick for diff-aware-pr-review: anthropic, scores as of 2026-10-19. Agents: GET /api/decide?jtbd=diff-aware-pr-review