Ephemeral CI runners for agent jobs
Spin isolated CPU/GPU sandboxes per CI job to run agent integration tests, tool mocks, and eval suites that need real browsers or kernels.
At a glance — Oct 2026
Updated 2026-10-27 · forward this page: /jtbds/agent-ephemeral-ci#briefScores as of 2026-10-19 (8 days old).
- Depot Pick
- Modal · 8.63/10 for this job (curated estimate, high confidence)
Serverless GPU/CPU job DX for CI. Confidence: high for Modal; Box still preview.
- Runner-up
- E2B · 8.70/10
Scores higher than the pick on: Capability fit, Privacy / data retention, Lock-in risk (higher = worse).
- When not to use the pick
- Need browser-native sandboxes (E2B), Fly Machines already in mesh, or Upstash Box when GA fits.
- Lock-in / migration
- Runner-up E2B has lower stored lock-in risk (4 vs 4.5, 0–10, higher = more lock-in). Self-host fallback listed for this job: Dagger.
- Sourced price (no comparable unit)
Pick: no sourced price in this job's unit (no_matching_sourced_price).
No cheapest SKU (no_candidate_has_sourced_price_in_job_unit:vcpu_hour). No price was invented.
- Free option
- Dagger (oss) — source, as of 2026-10-25
- Failure modes to watch
- Benchmark protocol mismatch: Store assisted-or-not, subset, and harness version beside every cited score. Refuse a ranking that mixes those fields.
- Collateral side effects: Snapshot the workspace or app state, then diff it against an allowlist. A passing check plus an unexpected diff is a failure.
- Context overflow: If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.
- Eval contamination: Keep gold labels out of the agent prompt. Store the harness version beside the score. Do not compare a number across a benchmark revision without saying the revision changed.
- Hallucinated tool arguments: Reject a call that fails the current schema or names an unknown id, and do that before the tool runs a side effect.
- Instruction drift: Restate the stop condition in the trace. End the run when it is met. Review long traces for steps that do not serve the original task.
Cite: Agent Provider Depot, Depot Pick for agent-ephemeral-ci: modal, scores as of 2026-10-19. Agents: GET /api/decide?jtbd=agent-ephemeral-ci
Free Agent CI runners options · for coding agents
Modal
Serverless GPU/CPU job DX for CI. Confidence: high for Modal; Box still preview.
Pricing (directional): As of 2026-10-24 (modal.com/pricing, fetched 2026-10-24): Python-native serverless GPU/CPU billed per second of compute ('You never pay for idle resources'). GPU list rates per second: H100 SXM5 '$0.001097', H200 SXM '$0.001261', B200 '$0.0…
Lock-in risk: higher = more lock-in (worse). Overall already inverts this dimension.
Depot darling for Python agent backends and bursty GPU jobs. If you think in functions not VMs, start here.
Quote for Ephemeral CI runners for agent jobs
Depot Pick stays Modal. Prices below are sourced SKU rows only (as of 2026-10-27). Input and output are not blended. GET /api/quote?jtbd=agent-ephemeral-ci
Nothing was compared. Pick has no sourced usd_per_vcpu_hour row. GPU-hour, GiB-hour, and vCPU-second were not converted. Cheapest stays null. pickIsPriceWinner is null.
- CPU physical core usd_per_core_second $0.0000131 source as of 2026-10-27
- Memory usd_per_gib_second $0.00000222 source as of 2026-10-27
- Network egress usd_per_gib $0.04 source as of 2026-10-27
- Nvidia A10 usd_per_gpu_second $0.000306 source as of 2026-10-27
- Nvidia A100 40GB usd_per_gpu_second $0.000583 source as of 2026-10-27
- Nvidia A100 80GB usd_per_gpu_second $0.000694 source as of 2026-10-27
- Nvidia B200 usd_per_gpu_second $0.001736 source as of 2026-10-27
- Nvidia B300 usd_per_gpu_second $0.001972 source as of 2026-10-27
- Nvidia H100 SXM5 usd_per_gpu_second $0.001097 source as of 2026-10-27
- Nvidia H200 SXM usd_per_gpu_second $0.001261 source as of 2026-10-27
- Nvidia L4 usd_per_gpu_second $0.000222 source as of 2026-10-27
- Nvidia L40S usd_per_gpu_second $0.000542 source as of 2026-10-27
- Nvidia RTX PRO 6000 usd_per_gpu_second $0.000842 source as of 2026-10-27
- Nvidia T4 usd_per_gpu_second $0.000164 source as of 2026-10-27
- Sandbox CPU usd_per_core_second $0.00003942 source as of 2026-10-27
- Sandbox memory usd_per_gib_second $0.00000667 source as of 2026-10-27
- Team plan usd_per_month $250 source as of 2026-10-27
- Volumes usd_per_gib_month $0.09 source as of 2026-10-27
1 of 5 candidates have no sourced price row. 5 are excluded from the USD per vCPU-hour comparison (vcpu_hour). Pick has no sourced usd_per_vcpu_hour row. GPU-hour, GiB-hour, and vCPU-second were not converted. Cheapest stays null.
Runners-up
E2B
Depot default code sandbox for agents. Use for untrusted code; Daytona when you need persistent…
Fly Machines
DIY sandbox substrate with global edge. Prefer E2B unless you need custom machine control.
Daytona
Choose when agents need sticky filesystems and full dev environments. E2B for disposable code exec. Confirm…
Upstash Box
Interesting E2B peer for freezeable agent computers. Keep medium until preview exits; production default…
When NOT to use the top pick
Need browser-native sandboxes (E2B), Fly Machines already in mesh, or Upstash Box when GA fits.
Full ranking (score table)
Ranked with this JTBD's weights. Scores are curated estimates unless marked otherwise. Agents: prefer /api/decide then this table for tie-breaks.
| # | Provider | Overall | Fit | $/perf | Latency | DX | As of |
|---|---|---|---|---|---|---|---|
| 1 | E2B | 8.7 | 9.4 | 8.0 | 8.6 | 9.4 | 2026-09-14 |
| 2 | ModalDepot Pick | 8.6 | 9.0 | 8.2 | 8.6 | 9.6 | 2026-10-19 |
| 3 | Daytona | 8.3 | 8.8 | 8.2 | 8.2 | 8.8 | 2026-09-14 |
| 4 | Fly Machines | 8.2 | 8.5 | 8.5 | 8.2 | 8.0 | 2026-09-14 |
| 5 | Upstash Box | 7.8 | 8.0 | 8.6 | 7.2 | 8.8 | 2026-09-14 |
Score methodology for this job
Serverless GPU/CPU job DX for CI. Confidence: high for Modal; Box still preview.
capabilityFit: 22%pricePerformance: 14%latencyReliability: 14%developerDx: 28%privacyPosture: 8%lockInRisk: 6%maturity: 8%