Tool-calling production agents

Scores as of 2026-10-19 (8 days old).

Depot Pick

Anthropic · 8.42/10 for this job (curated estimate, high confidence)

Capability + reliability + dx. Prefer first-party APIs over routers for SLA-critical tool loops.

Runner-up

OpenAI · 8.22/10

Scores higher than the pick on: Latency / reliability, Developer & agent DX, Maturity / track record.

When not to use the pick

You need OpenAI Realtime/multimodal tools specifically, or a multi-provider router during migration (OpenRouter).

Lock-in / migration

The pick has lower stored lock-in risk than runner-up OpenAI (6.5 vs 7.2). Self-host fallback listed for this job: MCP Python SDK, MCP reference servers, OpenAPI Specification.

Sourced price (usd_per_million_input_tokens)

Pick's lowest sourced SKU: Haiku 5.5 (prompts up to 100K tokens) $0.1 (usd_per_million_input_tokens) — source, as of 2026-10-27, re-checked 2026-10-27

Cheapest matching SKU: Anthropic Haiku 5.5 (prompts up to 100K tokens) $0.1 — source, as of 2026-10-27, re-checked 2026-10-27. The pick is the cheapest.

Price history: first seen 2026-10-21 · last changed no change recorded · history JSON

Free option

• Berkeley Function Calling Leaderboard (oss) — source, as of 2026-10-25
• GitHub MCP server (oss) — source, as of 2026-10-25
• MCP Python SDK (oss) — source, as of 2026-10-25

Failure modes to watch

• Hallucinated tool arguments: Reject a call that fails the current schema or names an unknown id, and do that before the tool runs a side effect.
• Runaway spend: Set a budget and a tool-call cap per task. Alert when spend jumps. Stop the run at the cap instead of summarizing a partial trace as success.
• Schema drift: Validate every tool call and every loaded table against the current schema. Quarantine mismatches instead of coercing them.
• Tool loop: Hash the last several tool names and arguments. Stop when the same call repeats and the observation did not change. Cap tool calls per task.

Reported outcomes (submitted reports only)

This job: loading… JSON · reports never change scores · seed/test records excluded

Depot Pick (Anthropic) on this job: loading… JSON · reports never change scores · seed/test records excluded

Same-job, same-unit comparisons: Anthropic vs OpenAI · Anthropic vs Google AI (Gemini) · Anthropic vs OpenRouter

Cite: Agent Provider Depot, Depot Pick for agent-tools: anthropic, scores as of 2026-10-19. Agents: GET /api/decide?jtbd=agent-tools