{"id":"llm-tracing","slug":"llm-tracing","name":"LLM/agent tracing","shortName":"LLM/agent tracing","problemStatement":"See prompts, tool calls, latency, and failures in production.","whenNotTopPick":"Deep LangChain shop — LangSmith. Want proxy in 10 minutes — Helicone. OSS-only — Phoenix. Python/PydanticAI traces + app OTel together — Logfire. Free eval+trace SKU — HoneyHive.","categoryIds":["cat-observability"],"depotPickProviderId":"langfuse","depotPickRationale":"dx + pricePerf + privacy (self-host option).","runnersUpIds":["helicone","langsmith","arize-phoenix","portkey","langwatch","comet-opik","logfire","honeyhive"],"weights":{"capabilityFit":0.2,"pricePerformance":0.12,"latencyReliability":0.12,"developerDx":0.12,"privacyPosture":0.25,"lockInRisk":0.09,"maturity":0.1},"methodologyNotes":"dx + pricePerf + privacy (self-host option).","tags":["llm"],"depth":{"whenNotToUse":{"text":"Deep LangChain shop — LangSmith. Want proxy in 10 minutes — Helicone. OSS-only — Phoenix. Python/PydanticAI traces + app OTel together — Logfire. Free eval+trace SKU — HoneyHive.","source":"editorial `whenNotTopPick` (data/jtbds.json)"},"whenNotToUseReason":null,"runnerUp":{"providerId":"helicone","name":"Helicone","overallForJob":7.94,"pickOverallForJob":8.62,"beatsPickOn":[],"source":"first resolvable entry of the job's editorially ranked `runnersUpIds`; scores are stored job-weighted scores"},"runnerUpReason":null,"lockIn":{"pickLockInRisk":2.5,"runnerUpLockInRisk":4,"scale":"0–10 stored curated estimate; higher = more lock-in (worse)","pickExitSignals":["open source (provider tag `oss`)"],"runnerUpExitSignals":[],"pickNotes":[{"text":"You operate it if self-hosting","field":"weaknesses"}],"selfHostFallback":[{"id":"langfuse-oss","name":"Langfuse self-host","license":"MIT Expat outside ee/","sourceUrl":"https://github.com/langfuse/langfuse/blob/main/LICENSE"}],"migration":"The pick has lower stored lock-in risk than runner-up Helicone (2.5 vs 4). Pick exit signals: open source (provider tag `oss`). Self-host fallback listed for this job: Langfuse self-host.","migrationReason":null,"derivedFrom":"stored lockInRisk scores, provider tags (oss, self-host, openai-compat, open-models), free rows that list this job, and verbatim provider notes. No migration guide was invented."}},"freshness":{"pick":{"score":{"asOf":"2026-09-14","ageDays":43,"stale":true,"staleAfterDays":30,"reason":null},"lastReviewed":{"asOf":"2026-09-14","ageDays":43,"stale":true,"staleAfterDays":30,"reason":null},"sourceCheck":{"checkedAt":"2026-10-27","sourceUrls":1,"sourcesReachable":1,"pricingRows":0,"pricingRowsReconfirmed":0,"note":"A source re-check confirms the page is reachable and, for pricing rows, that the stored amount and SKU name appear on it. It does not re-derive or change the score.","reason":null}}},"depotPick":{"id":"langfuse","slug":"langfuse","name":"Langfuse","tagline":"Depot default observability for most agent teams: open, affordable, good enough to serious.","overall":8.62},"runnersUp":[{"id":"helicone","slug":"helicone","name":"Helicone","tagline":"Maintenance mode since 2026-03-03 (Mintlify); site now says 'no longer part of Mintlify'. Runner-up only, not a pick.","overall":7.94},{"id":"langsmith","slug":"langsmith","name":"LangSmith","tagline":"If you live in LangGraph, LangSmith is the obvious observatory. Otherwise compare Langfuse/Helicone first.","overall":7.92},{"id":"arize-phoenix","slug":"arize-phoenix","name":"Arize Phoenix","tagline":"Choose Phoenix when you want open-source observability you can own. Cloud Arize when enterprise wants a…","overall":8.4},{"id":"portkey","slug":"portkey","name":"Portkey","tagline":"Depot pick for production AI gateway (fallbacks, virtual keys, budgets). OpenRouter when you want a model…","overall":8.18},{"id":"langwatch","slug":"langwatch","name":"LangWatch","tagline":"Strong when you need multi-turn/voice agent simulations with EU self-host. Langfuse default tracing…","overall":8.06},{"id":"comet-opik","slug":"comet-opik","name":"Comet Opik","tagline":"Pick when you already live in Comet or want OSS Opik + optional cloud. Langfuse remains cheaper OSS default…","overall":7.92},{"id":"logfire","slug":"logfire","name":"Pydantic Logfire","tagline":"Best when the agent is already PydanticAI/Python and you want app+LLM traces together. Langfuse stays Depot…","overall":8.26},{"id":"honeyhive","slug":"honeyhive","name":"HoneyHive","tagline":"Use the free Developer tier to stand up agent evals/traces in a day; Enterprise when you need self-host…","overall":8.07}]}