Jobs To Be Done
Primary navigation is by outcome — what your agent is trying to achieve — not by vendor marketing categories.
Frontier general chat & reasoning
Need the strongest general-purpose LLM API for product chat, planning, and hard reasoning.
Coding agent / code generation
Agent must edit repos, call tools, and produce correct code with high reliability.
Long-context agent workflows
Need 200K–1M context for large docs, codebases, or multi-hour agent state.
Tool-calling production agents
Reliable structured tool use, parallel tools, and low schema-slip in production agents.
High-volume cheap LLM inference
Millions of tokens/day for classification, extraction, or drafts where frontier quality is optional.
Hosted open-weight model inference
Run Llama/Qwen/DeepSeek/etc. without owning GPUs, with production DX.
Ultra-low-latency token streaming
Voice/realtime UX needs extremely fast tokens/sec on supported models.
Multi-provider model routing
One integration to A/B models and fail over across vendors.
Enterprise-governed LLM access
Procurement, VPC, IAM, BAA, and regional residency dictate cloud wrappers.
EU-friendly / sovereign LLM
Prefer EU vendor posture and data residency options for LLM APIs.
Embeddings for RAG
High-quality embeddings (+ optional rerank) for retrieval pipelines.
Enterprise RAG stack
Managed retrieval quality with rerank, security, and support.
Multimodal understanding (vision+)
Agents must reason over images/video/audio inputs at scale.
Web-grounded Q&A with citations
Answers must reflect live web with citations without DIY search stack.
Serverless GPU inference/jobs
Run GPU work with scale-to-zero and minimal ops.
Cheapest on-demand GPUs
Minimize $/GPU-hr for fault-tolerant training/batch.
Predictable dedicated GPU capacity
Labs need known capacity and clusters without marketplace chaos.
Multi-node GPU training
Train/fine-tune large models across many GPUs with fast interconnect.
Enterprise GPU cloud
SOC2, dedicated capacity, K8s, procurement-friendly GPU estate.
Self-hosted production inference fleet
Always-on model serving on rented GPUs with control over stacks.
Batch / interruptible GPU jobs
Offline evals, embeddings, or training steps that tolerate preemption.
Fine-tune open models
SFT/DPO/LoRA without building training infra from scratch.
Host agent web app / API
Deploy the product API and UI with minimal ops.
Long-running agent worker backend
Workers that run longer than serverless HTTP timeouts.
Global edge deploy for agent gateways
Low-latency thin agent gateways and tool endpoints worldwide.
Cheap always-on VMs
Minimize cost for 24/7 CPU boxes (bots, queues, reverse proxies).
Enterprise general compute
VMs inside an existing hyperscaler estate with compliance.
Managed vector database
Zero-ops vector search for production RAG.
Self-host vector DB
Run vectors in your VPC/K8s with open source.
Postgres-native vectors (pgvector)
Keep vectors next to relational data with minimal new systems.
Agent RAG memory store
Persistent retrieval memory for agents across sessions.
Cost-efficient large vector corpora
Store/search very large or cold-heavy vector sets without Pinecone-scale bills.
Billion-scale vector search
ANN at enormous scale with serious indexing.
Prototype RAG in a day
Hackathon/demo RAG with minimal setup.
Hot agent KV / session memory
Sub-ms session state, caches, and short-term memory.
LLM/agent tracing
See prompts, tool calls, latency, and failures in production.
Prompt/agent eval harness
Regression-test agent quality in CI before ship.
LLM cost monitoring
Track token spend per user/feature and catch bill spikes.
ML experiment tracking
Track training runs, metrics, artifacts for fine-tunes.
Infra-wide agent observability
Debug distributed agent systems beyond prompts (queues, gateways, VMs).
Cloud browser for agents
Give agents Playwright-grade browsers with stealth/sessions managed.
Self-host browser automation
Run Chromium fleets in your infra.
Web scraping / page fetch
Fetch and clean web content for agents.
LLM-ready web extraction
Turn sites into markdown/structured JSON for RAG.
SERP / search results API
Programmatic search engine results pages.
Enterprise proxy & unlocker network
Bypass tough anti-bot at scale with compliant enterprise tooling.
Autonomous web-acting agent
Agent logs in, clicks, fills forms, completes workflows in browsers.
Secure code sandbox for agents
Execute untrusted LLM-generated code safely.
Code interpreter tool for LLMs
Give any model a sandboxed REPL without OpenAI lock-in.
Persistent agent dev environments
Agents need sticky workspaces with files/git across sessions.
Transactional email
Send receipts, magic links, agent notifications reliably.
Marketing / blast email
Newsletters and campaigns (not pure transactional).
Programmable SMS
Send/receive SMS for agents and auth.
Programmable voice / phone agents
Inbound/outbound voice connected to LLM agents.
WhatsApp messaging
Agent messaging over WhatsApp Business APIs.
Multi-channel notification orchestration
Route email/SMS/push/Slack with preferences and templates.
Product authentication
User signup/login/organizations for an agent product.
Enterprise SSO / SCIM
Close enterprise deals that require Okta/Azure AD SSO and directory sync.
User management UI & orgs
Prebuilt user profiles, orgs, and session management.
Payments acceptance
Charge customers for the agent product.
Subscription & usage billing
Seat + metered token/API billing for AI products.
Programmatic image generation
Generate images from agents/apps via API.
Programmatic video generation
Generate short videos via API for products/agents.
Unified media generation API
One API across many image/video/audio models.
Text-to-speech
Natural TTS for agents and products.
Speech-to-text / transcription
Transcribe audio for agents.
Voice agents (talking products)
Build agents that speak and listen on phone/web.
3D generation
Generate 3D assets from prompts/images.
Web search tool for agents
Give agents a search tool that returns clean results for reasoning.
Neural / semantic web search
Find relevant pages by meaning for research agents.
Search-augmented generation
Combine search + LLM for fresher answers.
Privacy-oriented search API
Search with independent index and stronger privacy posture.
Enterprise web search API
Official search API under enterprise cloud terms.
Object storage for agent artifacts
Store files, datasets, and agent outputs.
Object storage without egress pain
Artifacts are downloaded often; S3 egress would dominate bill.
Serverless Postgres for agents
Scale-to-zero SQL with branching for previews/agents.
All-in-one app backend (DB/auth/storage)
Ship agent product backend without stitching five vendors.
Hybrid keyword + vector RAG
Need BM25/keyword + vector in one system.
Ultra-budget open-model inference
Need the cheapest OpenAI-compatible host for high-volume open weights without caring about brand prestige.
Edge-colocated LLM inference
Agent runs on edge/Workers and needs low-latency small/open models next to the runtime.
LLM on existing lakehouse data
Data already lives in Databricks or Snowflake; want governed inference/RAG without exporting corpora.
Self-hosted optimized model serving
Run optimized inference containers on your own NVIDIA GPUs across clouds.
Neocloud reserved / list-price GPUs
Need predictable H100/H200/Blackwell capacity with public list prices and cluster features.
Ultra-cheap interruptible GPUs
Batch/vision jobs that can restart; minimize $/GPU-hr on consumer or spot capacity.
Multi-cloud GPU shopping / routing
Compare and provision GPUs across many clouds from one API without N consoles.
SaaS tool / MCP router for agents
Connect agents to many third-party apps (Gmail, GitHub, CRMs) with managed OAuth/MCP.
Computer-use / stealth cloud browsers
Agents must operate authenticated, bot-hostile sites with human-like browsers.
Managed browser-agent loop
Want NL goal → browser agent with optional raw CDP, not just a session host.
Sandbox + full app platform
Agents need isolated compute plus APIs, DBs, GPUs, or BYOC — not only a code jail.
Hosted agent memory layer
Bolt persistent memory (add/retrieve) onto any agent without building a vector stack.
Temporal / graph agent memory
Agents must recall entities, relationships, and time-evolving facts — not just similar chunks.
Stateful long-lived agent runtime
Need agents with editable memory blocks and long-lived identity, not just a chat API + DB.
Embedded / object-store vector DB
Want local or object-storage-centric vectors without a heavy managed cluster.
OSS / CI-native LLM evals
Run prompt evals and red teams in CI without paying for a hosted eval SaaS.
RAG-specific evaluation metrics
Measure faithfulness, context precision/recall for retrieval pipelines.
Production guardrails & hallucination monitoring
Detect and block bad outputs in production with research-grade metrics.
Ultra-low-latency text-to-speech
Voice agents need streaming TTS with minimal time-to-first-audio.
Production speech-to-text
Accurate, cheap streaming and pre-recorded transcription for agents and voice apps.
Full voice-agent platform (STT+TTS+orchestration)
Ship a realtime voice agent without stitching STT, LLM, and TTS yourself.
Durable long-running agent workflows
Multi-step agents that must survive crashes, retries, and human waits over hours/days.
Event-driven agent jobs with great DX
Fan-out, debounce, and retry agent work from events without operating queues.
Hosted LangGraph agent deployment
Deploy LangGraph graphs with managed runtime and LangSmith traces.
Distributed Ray workloads for agents/ML
Scale Python agent/ML workloads across a Ray cluster with managed control plane.
Agent secrets & non-human credentials
Store and inject API keys for agents without leaking long-lived secrets into prompts, logs, or sandboxes.
Content moderation (I/O safety classify)
Classify user or model text/images for policy violations before acting.
Programmable LLM input/output guardrails
Enforce custom safety, topical, and tool-use policies around agent LLM calls — not just a fixed taxonomy.
AI usage metering & entitlements
Meter tokens/tools/API calls and enforce entitlements for usage-based agent SaaS billing.
People/company enrichment for research agents
Resolve emails, titles, and firmographics so research/GTM agents can act on real contacts.
Hosted desktop computer-use sandboxes
Give agents a full GUI desktop (mouse/keyboard/screenshot), not just a headless browser.
MCP server discovery & registries
Find, verify, and optionally host MCP tool servers for agents.
Progressive rollout of agent prompts, models, and tools
Ship a new prompt, model, or tool to 1% of agent traffic, eval online, and auto-rollback without a deploy.
SQL warehouse as agent context
Let agents query production analytics (SQL, Iceberg/DuckLake, shared semantic Guides) rather than dumping a warehouse into a vector DB.
Prompt-injection / jailbreak firewall
Screen untrusted user and retrieved content for instruction-override before it reaches the model — a dedicated firewall, not a content taxonomy.
Unified cloud + GPU + LLM FinOps
Allocate OpenAI/Anthropic/Modal/GPU spend per agent, customer, or feature next to the AWS bill — not just token traces in a gateway.
Serverless agent event backbone (queue/bus)
Need durable delayed HTTP callbacks, fan-out, or a real message fabric for agents — not a workflow runtime.
Discover and govern A2A/multi-agent registries
Need a registry to publish/find Agent Cards (A2A) and related MCP/skills across teams or the public internet — not just an MCP directory.
Self-host / OSS feature flags for agent rollouts
Need progressive delivery and kill-switches for agent tools/prompts without LaunchDarkly/Statsig seat economics — preferably OSS or cheap seats.
Hyperscaler queue/bus for agents already on AWS
Agent callbacks, fan-out, and cron need to live inside AWS IAM/VPC — not a third-party HTTP queue.
Production LLM AI gateway
Need a unified gateway in front of many model providers with fallbacks, virtual keys, budgets, guardrails, and request logs — not just a model marketplace.
Multilingual / code-switching speech-to-text
Agents must transcribe 20–50+ languages or mid-utterance language switches with production accuracy — not English-only streaming.
Scraper Actor / marketplace platform
Need a marketplace of reusable scrapers (Actors), scheduling, proxies, and agent-payable runs — not a single crawl-to-markdown API.
Cross-session / portable user memory
Need durable user-profile or portable memory that follows the user across sessions and assistants (often MCP), not just a vector DB or fact store.
Parse documents into LLM-ready chunks
Turn PDFs, Office, scans, and messy forms into markdown/JSON an agent can retrieve or extract from — not a web crawler.
Rerank retrieved chunks for RAG
After vector search, need a dedicated reranker so the agent sees the right 5–20 chunks — not just 'better embeddings'.
Real-device mobile computer-use
Agent must tap/swipe a real iOS or Android device (Appium/UIAutomator), not a desktop VM or headless Chrome.
Realtime agent↔user channel
Stream agent tokens, tool progress, and presence to clients over WebSocket/SSE with pub/sub semantics — not email/SMS and not a backend work queue.
Edge / per-tenant SQLite for agents
Need many lightweight per-user or per-agent SQL databases at the edge (libSQL/SQLite), not a single Postgres or Redis cache.
Sell AI/agent products as Merchant of Record
Need to charge for tokens, agent runs, seats, and credits with global tax/VAT handled — without building Stripe Tax + meters + invoicing yourself.
Serverless HTTP Redis from edge/agents
Need Redis-compatible KV/cache/rate-limits from Workers/Vercel/Lambda without TCP connection pools or always-on memory bills.
Structured web → knowledge graph extract
Need schema'd entities/products/articles from the public web (not just markdown chunks) for agent grounding.
Reliable outbound webhooks for agent products
Need to deliver signed, retried webhooks to customers (agent run finished, invoice, tool events) without building a queue+portal yourself.
Embed SaaS OAuth + API sync for agents
Need customers to connect HubSpot/Jira/Notion/etc. with durable OAuth tokens and syncs — not just one-shot MCP tool calls.
High-volume agent event/log analytics warehouse
Need OLAP SQL over millions of agent traces, tool calls, and product events — not just an app Postgres or DuckDB laptop.
Application error/trace monitoring for agent backends
Need exception tracking, performance spans, and alerts for the services hosting agents — separate from LLM request tracing.
Confidential / TEE-backed LLM inference
Run inference where the host cannot read prompts/weights in cleartext — confidential VMs or TEE-backed serverless for regulated or high-trust agents.
High-volume cheap batch embeddings
Embed millions of chunks overnight where per-token quality matters less than $/1M and throughput; async/batch APIs preferred.
Continuous eval gates in CI/CD
Block merges when agent prompts/tools regress — hosted eval suites, datasets, and pass/fail gates wired to GitHub/GitLab CI.
Prompt-caching cost optimization
Cut spend on long system prompts, tool schemas, and RAG prefixes via provider prompt caching / context caching with predictable hit rates.
PSTN / phone-number telephony for agents
Provision numbers, place/receive PSTN calls, and bridge SIP/WebRTC so voice agents can dial real phones — not just in-browser audio.
Calendar & scheduling tools for agents
Agents must read/write Google/Microsoft calendars, propose slots, and book meetings via OAuth toolkits without building calendar OAuth from scratch.
Layout-aware OCR & form extraction
Extract tables, checkboxes, multi-column layouts, and key-value forms from PDFs/scans where naive chunking destroys structure.
Spreadsheet / CSV agents with SQL
Let agents query uploads, Google-sheet exports, and warehouse-like CSVs with SQL + LLM planning instead of brittle cell scraping.
Diff-aware pull-request review agents
Review multi-file diffs with high tool-use reliability, low false-positive nits, and safe suggestions for CI review bots.
Automated secret rotation for agents
Rotate API keys, DB creds, and workload identities on a schedule with zero-downtime dual-publish so long-running agents do not die mid-job.
LLM audit-log / compliance warehouse
Retain prompt/tool/decision logs for SOC2/ISO auditors in a queryable warehouse with retention, access control, and export — not just a tracing UI.
Synthetic evaluation & scenario data
Generate diverse agent test scenarios, adversarial prompts, and labeled trajectories without waiting on production traffic.
LoRA serving & distillation hosting
Train adapters or distill smaller students, then serve many LoRAs efficiently without one-GPU-per-customer economics.
Ephemeral CI runners for agent jobs
Spin isolated CPU/GPU sandboxes per CI job to run agent integration tests, tool mocks, and eval suites that need real browsers or kernels.
Screenshot & visual regression QA
Capture screenshots across browsers/devices and diff UI regressions for agent-driven release gates.
Form-filling & portal RPA agents
Fill complex government/enterprise web forms with vision+DOM agents when no API exists — retries, selectors, and human-hand-off included.
Hosted MCP server runtime
Deploy and scale MCP servers (tools/resources) with auth, logging, and HTTPS without babysitting containers — discovery alone is not enough.
Machine-to-machine agent identity
Issue non-human identities, mTLS/JWT, and scoped tokens so agents authenticate to APIs without sharing user sessions.
LLM rate-limit & backoff shield
Protect upstream model quotas with per-tenant limits, smart retries, fallbacks, and 429 shielding so one noisy agent cannot melt the budget.
Idempotency keys for agent side effects
Exactly-once-ish side effects (charges, tickets, emails) across retries using durable workflows and idempotency keys agents can pass end-to-end.
Dead-letter queues for agent tasks
Park failed agent runs with full payload, replay, and poison-pill isolation so human operators can unblock queues without data loss.
Human-in-the-loop approval workflows
Pause agents for human approve/reject on risky tools (spend, delete, send), with timeouts, escalations, and audit trails.
LLM spend anomaly alerts
Detect sudden $/day spikes, runaway tool loops, and model-mix regressions — alert Slack/PagerDuty before the invoice lands.
Low-carbon / region-aware GPU inference
Prefer cleaner energy regions or sustainable GPU clouds when agents can trade a few ms of latency for lower carbon intensity.
HIPAA-friendly LLM with BAA path
Process PHI-adjacent text with a provider willing to sign a BAA / equivalent and offer enterprise isolation controls.
Air-gapped / VPC model serving
Serve optimized models inside VPC/air-gapped networks with no mandatory data egress to public LLM APIs.
Edge / WASM-adjacent small-model inference
Run small LLMs/embedders next to users on edge runtimes for privacy and TTFT — not full frontier chat.
Knowledge-graph / Graph-RAG memory
Build entity-relation memory so agents answer multi-hop questions better than flat vector RAG alone.
Late-interaction / ColBERT-style retrieval
Need multi-vector late interaction retrieval for high-recall RAG where single-vector ANN misses nuanced matches.
Agent session trace replay
Replay full agent sessions (LLM calls, tools, browser steps) for debugging incidents and training reviewers — like session replay for agents.
Runtime LLM firewall / AI WAF
Inline policy enforcement on inputs/outputs (PII, jailbreaks, topic blocks) as a firewall layer in front of every model call.
Adversarial red-team evaluation suites
Continuously attack prompts/agents with jailbreak corpora and safety suites before and after each release.
Dataset & experiment versioning for agents
Version eval sets, traces-as-data, and fine-tune corpora with lineage so agent improvements are reproducible.
Vector + object-storage hybrid memory
Store embeddings cheaply atop object storage while keeping raw blobs (PDFs, images) co-located for agent re-read without double egress bills.
Long-term episodic agent memory
Remember user episodes across weeks with temporal validity (what was true when) — not just a static profile vector.
Inbound webhook gateway for agents
Receive third-party webhooks reliably (verify, dedupe, queue, observe) and wake agents without exposing raw public endpoints.
Deep research & multi-hop web agents
Multi-step web research with citations, extractions sources, and agent-friendly APIs — beyond single-shot search.
Text-accurate image generation
Generate marketing/UI images where spelled text on the image must be correct (posters, mockups, diagrams).
Mobile push notifications for agents
Notify humans on iOS/Android when agents need input or finish long jobs — without building APNs/FCM infrastructure.
Budget on-prem-ish batch open-model jobs
Run large offline batches on cheap distributed/consumer GPU networks when latency SLOs are hours not milliseconds.