Multi-Turn Synthetic User & Adversarial Persona Simulation
Stress-test voice and chat agents against hundreds of realistic, frustrated or adversarial synthetic personas in multi-turn conversations before a production rollout, with scored transcripts as the release gate.
At a glance — Oct 2026
Updated 2026-10-28 · forward this page: /jtbds/synthetic-persona-pre-flight#briefEditorial prior (2026-10-19) + deterministic source check (verified) + Countersign receipts (cs-54379c97, cs-6eab5b5d, cs-316f2587).
- Depot Pick
- Braintrust · 8.29/10 for this job (curated estimate, high confidence)
Braintrust runs multi-turn simulated-user evals with scorers and datasets as a CI quality gate, and keeps the transcripts comparable across releases.
- Runner-up
- Promptfoo · 8.43/10
Scores higher than the pick on: Price / performance, Privacy / data retention, Lock-in risk (higher = worse).
- When not to use the pick
- Open-source CLI red-teaming in CI — promptfoo; hallucination/judge models — Patronus; enterprise eval observability — Galileo.
- Lock-in / migration
- Runner-up Promptfoo has lower stored lock-in risk (2.5 vs 5, 0–10, higher = more lock-in).
- Sourced price (no comparable unit)
Pick: no sourced price in this job's unit (no_sourced_price).
No cheapest SKU (no_comparable_unit_for_job). No price was invented.
- Free option
- No free resource lists this job. A cheap paid SKU is not a free option.
- Failure modes to watch
- No failure mode plainly applies to this job (no_failure_mode_plainly_applies).
Cite: Agent Provider Depot, Depot Pick for synthetic-persona-pre-flight: braintrust, scores as of 2026-10-28. Agents: GET /api/decide?jtbd=synthetic-persona-pre-flight
Braintrust
Braintrust runs multi-turn simulated-user evals with scorers and datasets as a CI quality gate, and keeps the transcripts comparable across releases.
Pricing (directional): Eval-first platform; data-based pricing with unlimited seats on many plans; strong CI eval story.
Top pick (8.29) and #2 runner-up (8.43) are separated by 0.14 points (<= 0.35 weight-sensitivity band). Treat them as co-picks and use the workload tie-breaker rule below rather than the raw composite decimal.
Workload Tie-Breaker Rule: Open-source CLI red-teaming in CI — promptfoo; hallucination/judge models — Patronus; enterprise eval observability — Galileo.
Lock-in risk: higher = more lock-in (worse). Overall already inverts this dimension.
Depot pick when evals are the product quality gate. Pair with Helicone/Langfuse for cheap request logs if needed.
Quote for Multi-Turn Synthetic User & Adversarial Persona Simulation
Depot Pick stays Braintrust. Prices below are sourced SKU rows only (as of 2026-10-28). Input and output are not blended. GET /api/quote?jtbd=synthetic-persona-pre-flight
Nothing was compared. no comparable sourced unit for this job; chat input $/1M and other units were not borrowed pickIsPriceWinner is null.
Sourced price unavailable for the Depot Pick. No row in the pricing index.
3 of 4 candidates have no sourced price row. 4 are excluded from the Matching price comparison (none). no comparable sourced unit for this job; chat input $/1M and other units were not borrowed
Runners-up
Promptfoo
Default OSS eval harness. Braintrust when you want hosted collaboration; Promptfoo when you want local/CI…
Patronus AI
Layer on when regulated hallucination/safety gates matter more than general CI evals. Lakera for injection…
Galileo
Pick for production hallucination measurement + guardrails. Braintrust/Promptfoo for lighter eval loops.
When NOT to use the top pick
Open-source CLI red-teaming in CI — promptfoo; hallucination/judge models — Patronus; enterprise eval observability — Galileo.
Full ranking (score table)
Ranked with this JTBD's weights. Scores are curated estimates unless marked otherwise. Agents: prefer /api/decide then this table for tie-breaks.
| # | Provider | Overall | Fit | $/perf | Latency | DX | As of |
|---|---|---|---|---|---|---|---|
| 1 | Promptfoo | 8.4 | 8.2 | 9.2 | 7.8 | 8.8 | 2026-10-28 |
| 2 | BraintrustDepot Pick | 8.3 | 9.2 | 7.5 | 8.6 | 8.8 | 2026-10-28 |
| 3 | Galileo | 7.5 | 8.4 | 6.2 | 8.0 | 7.8 | 2026-10-28 |
| 4 | Patronus AI | 7.4 | 8.4 | 5.5 | 8.0 | 7.2 | 2026-10-28 |
Score methodology for this job
Scores multi-turn simulation support, scorer flexibility and CI-gate ergonomics.
capabilityFit: 26%pricePerformance: 16%latencyReliability: 16%developerDx: 16%privacyPosture: 12%lockInRisk: 6%maturity: 8%