Agent classes / Support agent / Content moderation (I/O safety classify)
Content moderation (I/O safety classify) for a support agent
Updated 2026-10-27 (newest stored date on this page) · score asOf 2026-09-14 (stale)
The class: Answers a customer from the operator's policy and escalates when sources disagree or the action is consequential.
- One canonical policy page per topic
- A human approval step for refunds and exceptions
Depot Pick: OpenAI Moderation
When not to use it: Need open Llama Guard classifier with low latency — groq; need programmable multi-step rails — nvidia-nemo-guardrails; need EU-hosted guard models — ovhcloud-ai.
Runner-up: Groq.
Cheapest matching sourced SKU (usd_per_1k_requests): OpenAI Moderation omni-moderation-latest (moderation endpoint) $0 — source, asOf 2026-10-21. All sourced prices
Failure modes this class and job share (10)
- Confirmation skip: Gate those tool names on an approval id recorded before the call. A missing approval is a failure even if the downstream tool returns success.
- Identity confusion: Bind every tool call to a user or tenant id. Refuse the call if that id changed mid-task. For a public bot, the operator owns the words the bot shows.
- Non-determinism: Run the task more than once and compare the end state, not a single transcript. Record the trial count next to any success rate you cite.
- Over-refusal: Log refusals separately from successes. Review a sample against the written policy. Do not treat a refusal as a pass, and do not weaken a safety check to raise the pass rate.
- Policy conflict: If cited sources contradict each other, return the conflict and the URLs. Do not pick a winner silently.
- Prompt injection: Compare each tool call to the original user task. If the call only makes sense given text that arrived from a fetched page, file, or message, stop and ask. Record which source the instruction came from. This catalog does not store payloads or attack steps.
- Schema drift: Validate every tool call and every loaded table against the current schema. Quarantine mismatches instead of coercing them.
- Stale retrieval: Store the source URL and the time it was fetched. If a newer canonical page exists, or two stored sources disagree, do not answer from the older one.
- Ungrounded answer: Require a source URL or passage id on factual sentences. Drop or mark any sentence whose source does not contain the claim.
- Weak verification: Success requires an end-state check the agent does not grade for itself: tests, a database diff, or a separate judge that cannot see the gold label.
Free options this class uses (0)
No free row lists this job with a kind this class needs.