Agent classes / Local and offline agent / Budget batch open models
Budget batch open models for a local and offline agent
Updated 2026-10-27 (newest stored date on this page) · score asOf 2026-10-19
The class: Runs without calling a hosted model API, using local weights, a local index, or an air-gapped serve path.
- No network call on the answer path
- Say when the local corpus has no support
Depot Pick: SaladCloud
When not to use it: Need predictable enterprise GPU (CoreWeave/Nebius) or TEE (Chutes).
Runner-up: Vast.ai.
Cheapest matching sourced SKU (usd_per_gpu_hour): SaladCloud RTX 3090 24GB (Lowest priority, batch) $0.09 — source, asOf 2026-10-19. All sourced prices
Failure modes this class and job share (6)
- Context overflow: If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.
- Lost in the middle: Ask the agent to name the passage it used. Spot-check questions whose only support sits in the middle of the context, not only at the edges.
- Non-determinism: Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.
- Schema drift: Validate every tool call and every loaded table against the current schema. Quarantine mismatches instead of coercing them.
- Silent truncation: Tools must return whether the payload was truncated and how long the original was. Fail the task if the answer depends on the missing span.
- Stale retrieval: Store the source URL and the time it was fetched. If a newer canonical page exists, or two stored sources disagree, do not answer from the older one.