Agent classes / Local and offline agent / Edge small models
Edge small models for a local and offline agent
Updated 2026-10-27 (newest stored date on this page) · score asOf 2026-09-14 (stale)
The class: Runs without calling a hosted model API, using local weights, a local index, or an air-gapped serve path.
- No network call on the answer path
- Say when the local corpus has no support
Depot Pick: Cloudflare Workers AI
When not to use it: Need LPU-class latency on larger open models (Groq), or Workers AI frontier SKUs require Workers Paid you will not buy.
Runner-up: Groq.
Cheapest matching sourced SKU (usd_per_million_input_tokens): Cloudflare Workers AI @cf/ibm-granite/granite-4.0-h-micro $0.017 — source, asOf 2026-10-03. All sourced prices
Failure modes this class and job share (6)
- Context overflow: If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.
- Lost in the middle: Ask the agent to name the passage it used. Spot-check questions whose only support sits in the middle of the context, not only at the edges.
- Non-determinism: Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.
- Schema drift: Validate every tool call and every loaded table against the current schema. Quarantine mismatches instead of coercing them.
- Silent truncation: Tools must return whether the payload was truncated and how long the original was. Fail the task if the answer depends on the missing span.
- Stale retrieval: Store the source URL and the time it was fetched. If a newer canonical page exists, or two stored sources disagree, do not answer from the older one.
Free options this class uses (1)
- Gemma models under the Gemma Terms of Use (open_weights, free) — source, asOf 2026-10-26