{"id":"cost-monitoring","slug":"cost-monitoring","name":"LLM cost monitoring","shortName":"LLM cost monitoring","problemStatement":"Track token spend per user/feature and catch bill spikes.","whenNotTopPick":"Need deep evals more than cost — Braintrust; need unified cloud+GPU+LLM bills not just token traces — Vantage; want a no-SDK drop-in proxy and accept maintenance-mode risk (since 2026-03-03, ownership unclear) — Helicone.","categoryIds":["cat-observability"],"depotPickProviderId":"langfuse","depotPickRationale":"Editorial replacement 2026-10-21: the former pick Helicone (8.13 for this job) has been in maintenance mode since 2026-03-03 (https://www.mintlify.com/blog/mintlify-acquires-helicone: 'Helicone will continue operating in maintenance mode') and its site now says 'Helicone is no longer part of Mintlify. Details to come soon.' (https://www.helicone.ai/, re-checked 2026-10-21), so ownership and roadmap are unclear. Replaced by the highest-scoring live runner-up, Langfuse (8.66 for this job; cost tracking per model/user/feature, spend-threshold alerts and a Metrics API per https://langfuse.com/docs/observability/features/token-and-cost-tracking, re-checked 2026-10-21). Helicone stays a runner-up with the maintenance_mode flag. No score changed.","runnersUpIds":["openrouter","vantage","portkey","helicone"],"weights":{"capabilityFit":0.25,"pricePerformance":0.18,"latencyReliability":0.15,"developerDx":0.15,"privacyPosture":0.1,"lockInRisk":0.07,"maturity":0.1},"methodologyNotes":"dx + pricePerf of the observability tool itself.","tags":["cost"],"depth":{"whenNotToUse":{"text":"Need deep evals more than cost — Braintrust; need unified cloud+GPU+LLM bills not just token traces — Vantage; want a no-SDK drop-in proxy and accept maintenance-mode risk (since 2026-03-03, ownership unclear) — Helicone.","source":"editorial `whenNotTopPick` (data/jtbds.json)"},"whenNotToUseReason":null,"runnerUp":{"providerId":"openrouter","name":"OpenRouter","overallForJob":8,"pickOverallForJob":8.66,"beatsPickOn":["developerDx"],"source":"first resolvable entry of the job's editorially ranked `runnersUpIds`; scores are stored job-weighted scores"},"runnerUpReason":null,"lockIn":{"pickLockInRisk":2.5,"runnerUpLockInRisk":4,"scale":"0–10 stored curated estimate; higher = more lock-in (worse)","pickExitSignals":["open source (provider tag `oss`)"],"runnerUpExitSignals":["OpenAI-compatible API (provider tag `openai-compat`)"],"pickNotes":[{"text":"You operate it if self-hosting","field":"weaknesses"}],"selfHostFallback":[],"migration":"The pick has lower stored lock-in risk than runner-up OpenRouter (2.5 vs 4). Pick exit signals: open source (provider tag `oss`).","migrationReason":null,"derivedFrom":"stored lockInRisk scores, provider tags (oss, self-host, openai-compat, open-models), free rows that list this job, and verbatim provider notes. No migration guide was invented."}},"freshness":{"pick":{"score":{"asOf":"2026-09-14","ageDays":43,"stale":true,"staleAfterDays":30,"reason":null},"lastReviewed":{"asOf":"2026-09-14","ageDays":43,"stale":true,"staleAfterDays":30,"reason":null},"sourceCheck":{"checkedAt":"2026-10-27","sourceUrls":1,"sourcesReachable":1,"pricingRows":0,"pricingRowsReconfirmed":0,"note":"A source re-check confirms the page is reachable and, for pricing rows, that the stored amount and SKU name appear on it. It does not re-derive or change the score.","reason":null}}}}