Promptfoo alternatives
Default OSS eval harness. Braintrust when you want hosted collaboration; Promptfoo when you want local/CI…
Overall 8.42 (curated_estimate, high) · score asOf 2026-10-19 · last reviewed 2026-10-19. This is not an Elo and not a live price board. Provider · JSON
Jobs: llm-tracing, evals, oss-evals, rag-evals, continuous-eval-ci, synthetic-eval-data, adversarial-red-team-evals
| Provider | Why listed | Overall | Δ vs subject | asOf |
|---|---|---|---|---|
| Braintrust Depot pick when evals are the product quality gate. Pair with Helicone/Langfuse for cheap request logs if… | same_jtbd_pick, same_jtbd_runner · evals, oss-evals, rag-evals | 8.23high | -0.19 | 2026-10-19 |
| Langfuse Depot default observability for most agent teams: open, affordable, good enough to serious. | same_jtbd_pick, same_jtbd_runner · llm-tracing, evals | 8.66high | +0.24 | 2026-09-14 |
| Ragas Default RAG metric library. Pair with Promptfoo/Braintrust for broader agent evals. | same_jtbd_runner, same_jtbd_pick · oss-evals, rag-evals | 8.33high | -0.09 | 2026-09-14 |
| OpenAI Default when you want maximum capability and ecosystem glue. Multi-home critical paths behind OpenAI-compat… | same_jtbd_pick · synthetic-eval-data | 8.22high | -0.2 | 2026-09-14 |
| LangSmith If you live in LangGraph, LangSmith is the obvious observatory. Otherwise compare Langfuse/Helicone first. | same_jtbd_runner · llm-tracing, evals, oss-evals | 8.09high | -0.33 | 2026-09-14 |
| Confident AI Best when your team already thinks in pytest/DeepEval. Braintrust remains Depot hosted-evals default… | same_jtbd_runner · evals, oss-evals, continuous-eval-ci | 7.99high | -0.43 | 2026-09-14 |
| LangWatch Strong when you need multi-turn/voice agent simulations with EU self-host. Langfuse default tracing… | same_jtbd_runner · llm-tracing, evals, oss-evals | 8.06high | -0.36 | 2026-09-14 |
| Comet Opik Pick when you already live in Comet or want OSS Opik + optional cloud. Langfuse remains cheaper OSS default… | same_jtbd_runner · llm-tracing, evals, oss-evals | 8.04high | -0.38 | 2026-09-14 |
| Arize Phoenix Choose Phoenix when you want open-source observability you can own. Cloud Arize when enterprise wants a… | same_jtbd_runner · llm-tracing, evals | 8.44high | +0.02 | 2026-09-14 |
| Pydantic Logfire Best when the agent is already PydanticAI/Python and you want app+LLM traces together. Langfuse stays Depot… | same_jtbd_runner · llm-tracing, evals | 8.33high | -0.09 | 2026-09-14 |
| HoneyHive Use the free Developer tier to stand up agent evals/traces in a day; Enterprise when you need self-host… | same_jtbd_runner · llm-tracing, evals | 8.02high | -0.4 | 2026-09-14 |
| Galileo Pick for production hallucination measurement + guardrails. Braintrust/Promptfoo for lighter eval loops. | same_jtbd_runner · rag-evals, adversarial-red-team-evals | 7.45high | -0.97 | 2026-09-14 |
Prices are not ranked on this page. For a job, use the sourced quote (separate input and output rows, Depot Pick unchanged): llm-tracing, evals, oss-evals, rag-evals.
Alternatives are other catalog providers that share a JTBD as pick or runner-up. If fewer than three exist, same-category active providers fill the list and are labeled same_category. Overall deltas are catalog scores. kind=curated_estimate is not a lab measurement. Pricing rows are sourced or explicitly null. Do not parse pricingNotes. A single normalized USD figure is set only for a provider with exactly one token row.