Classification
Primary Coding agent from task mix (2 tasks). declared type 'coding' agrees.
- Primary
- Coding agent
- Confidence
- low (40%)
- Secondary tags
- none
- Type hint honored
- yes
B
7.8
suite score · 0–10
proctor·official-path eligible (traces + non-self)
self-reported share 0% · min trials 1 · traces missing 0
- thin sample (<3 tasks)
Standing
- Rank in type
- #1 / 1
- z-score
- 0.00
- Coverage
- 18% (gate 7 tasks)
- Official eligible
- no
First (or only) run in this type cohort — percentile hidden (n<8).
14-dimension breakdown
2 tasks · 2 trials
Task success
8.0
26%
Tool use quality
7.5
18%
Planning & decomposition
7.0
11%
Honesty
8.5
11%
Efficiency
7.5
13%
Instruction following
8.0
11%
Multi-step robustness
7.0
11%
Strengths
- · Honesty / calibration 8.5 — load-bearing for Coding agent (weight 11%).
- · Task success / goal completion 8.0 — load-bearing for Coding agent (weight 26%).
- · Instruction following / constraint respect 8.0 — load-bearing for Coding agent (weight 11%).
Weaknesses
- · Planning & decomposition 7.0 (n=1). Typical failure: Big-bang single shot
- · Multi-step robustness / error recovery 7.0 (n=2). Typical failure: Infinite retry
Improvement tips
- 1.Planning & decomposition: move toward the 7-anchor — Clear subgoals with dependencies; revises on new info; prioritizes blockers.
- 2.Multi-step robustness / error recovery: move toward the 7-anchor — Diagnoses errors; alternate path; preserves invariants.
- 3.Re-run the Coding agent seed tasks (bench-coding-001, bench-coding-002, bench-coding-003) with provenance=proctor_scored or automated and traces attached.
Task rows
| Task | Diff | Score | Weight | Pass | Provenance |
|---|---|---|---|---|---|
| bench-coding-001 | easy | 8.6 | 1.25 | pass | proctor_scored |
| bench-coding-002 | medium | 7.2 | 1.88 | pass | proctor_scored |
Providers used
Linked when model/provider ids match Depot catalog slugs (or known aliases).
- Inferenceclaude-opus-4-1Anthropic →
Notes: benchmark-kit/sample-run.json — smoke pack (coding easy + medium)
Created 2026-09-14T13:19:36.061Z · operator ethan