Agent classes / RAG agent / ColBERT retrieval
ColBERT retrieval for a rag agent
Updated 2026-10-27 (newest stored date on this page) · score asOf 2026-09-14 (stale)
The class: Answers from a retrieved corpus and says when the corpus does not support the claim.
- Passage ids on every factual sentence
Depot Pick: Jina AI
When not to use it: Standard embed+rerank (Voyage/Cohere) meets quality bar, or Elastic already stores the corpus.
Runner-up: Voyage AI.
No single-unit sourced price comparison for this job (no_comparable_unit_for_job).
Failure modes this class and job share (7)
- Stale retrieval: Store the source URL and the time it was fetched. If a newer canonical page exists, or two stored sources disagree, do not answer from the older one.
- Context overflow: If the input exceeds the window, say so and retrieve a narrower slice. Do not answer from a truncated head as if it were the whole source.
- Eval contamination: Keep gold labels out of the agent prompt. Store the harness version beside the score. Do not compare a number across a benchmark revision without saying the revision changed.
- Lost in the middle: Ask the agent to name the passage it used. Spot-check questions whose only support sits in the middle of the context, not only at the edges.
- Policy conflict: If cited sources contradict each other, return the conflict and the URLs. Do not pick a winner silently.
- Silent truncation: Tools must return whether the payload was truncated and how long the original was. Fail the task if the answer depends on the missing span.
- Ungrounded answer: Require a source URL or passage id on factual sentences. Drop or mark any sentence whose source does not contain the claim.
Free options this class uses (0)
No free row lists this job with a kind this class needs.