The direct answer
What a backend for AI agents includes
A backend for AI agents is everything that stands between your product and a finished, reviewable result: the harness execution loop, the task and session lifecycle, streaming, file intake, artifacts back to your UI, routing, retries, and metering, with sandboxed execution included as one layer inside it.
The term carries two different meanings right now, so scope first. Some products use backend for AI agents to mean an app backend that AI builds or operates: databases, auth, and storage for an AI-made product. This article is about the other meaning: the agent backend, the runtime slice of your architecture that executes agent work. It sits beside the rest of your backend; nothing here replaces your API layer, your database, or your business logic.
When a product team adds an agent feature, the visible part is small: a user asks for work, and finished work appears. The invisible part is this backend, and it is easy to underestimate because its most talked-about piece, the sandbox, is only one of its layers.
This article lays out the complete set, then gives a practical way to decide, layer by layer, what to build and what to adopt. The framing is complementary: routing model calls is a model router's job, governing model traffic is an AI gateway's, and executing complete tasks is the backend's. Products at scale often use all three.
The problem
Why the sandbox-first mental model runs out
Many teams start with isolation, and for good reason: an agent doing real work takes real actions, it runs commands, writes and executes code, edits files, and keeping all of that away from your production systems is a genuine requirement that sandbox infrastructure answers well. The trap is concluding that once the work can execute safely, the backend is nearly done.
What surfaces next is everything around the execution: something has to run the agent loop inside that isolation, keep session state so a follow-up continues instead of restarting, stream progress and tool calls back to your product, move a user's files in, bring reviewable results out, retry what fails, and meter what ran. Little of that is the sandbox's job, and nearly all of it lands on your product the moment users touch the feature.
Teams that assemble this themselves often discover the shape of the work only as they go, because each surface looks small until it has to survive production. The sandbox turns out to be the best-understood layer precisely because it is the most contained one.
The mechanism
The layers of a complete agent backend
A complete backend for AI agents covers seven jobs:
- Harness execution: a complete agent loop that plans, calls tools, and edits files. Harnesses such as Codex, Claude Code, and Hermes are complete implementations you can adopt rather than write.
- Sandboxed execution: an isolated environment that keeps each task away from your production systems. In a complete agent backend this is included, not a separate purchase.
- Task and run lifecycle: submitting work, identifying each execution, and the observability to trace what happened.
- Sessions and memory: durable state so work can pause, resume, and continue, carrying context from one request to the next.
- Streaming: progress, events, and tool calls flowing back to your product while work runs.
- Files and artifacts: user inputs entering the workspace, and finished outputs returning as renderable artifacts your UI can present for human review.
- Routing, retries, and metering: choosing the harness and model for each task, recovering from failures, and accounting for what ran.
A fair question about this list: are several of these not already inside the harness? They are, in raw form. A complete harness streams its own events, keeps its own session state, reads and writes its own files. The backend's job is not to reinvent those; it is to turn each one into a durable, product-facing surface: a session your product can resume tomorrow, a stream your UI can subscribe to, a user's upload landing in the workspace and coming back as a reviewable artifact. That wrapping, plus the one thing no single harness can do, routing and metering across harnesses, is what makes it a backend.
The order matters less than the completeness: a production feature starts reaching these layers on day one, when the first user upload touches file intake, the first flaky run touches retries, and the first follow-up request touches sessions.
The decision
Build or adopt, decided layer by layer
The practical question is which path assembles these layers for you, and two paths anchor the spectrum. Assemble from primitives: pick a sandbox, wire an agent SDK into it, and build or finish the lifecycle, streaming, file, and routing layers around it yourself. Or adopt a complete agent backend: integrate one contract that already includes them.
The comparison below is between those two paths as architectural shapes, not between vendors. The primitive path is the right foundation when the agent system itself is your product and you intend to own every surface. The complete path fits when agents power features inside your product and the backend is means, not end.
| Capability | Assembled from sandbox primitives | Complete agent backend |
|---|---|---|
| Isolated execution | Core capability | Core capability |
| Complete harness execution loop | Varies | Core capability |
| Sessions and resumable state | Varies | Core capability |
| Streaming and tool-call events to your product | Varies | Core capability |
| File intake and artifacts back to your UI | Not this layer's primary job | Core capability |
| Multi-harness routing and retries | Not this layer's primary job | Core capability |
| Unified metering across harnesses | Not this layer's primary job | Core capability |
Read the marks as a division of labor, not a scorecard. Isolation providers are core at isolation because that is their layer's job; the layers above it are simply a different job, and on the primitive path that job is yours.
The bridge
What the complete path looks like in practice
HarnessRouter is the complete path behind one contract: your app sends one task; HarnessRouter runs the best harness in a sandbox and returns renderable artifacts to your UI. One integration covers harnesses like Codex, Claude Code, and Hermes, and the remaining jobs on the list, sessions, streaming, file intake, metering, ride the same contract instead of becoming separate projects.
The contract, the Unified Harness Protocol (UHP), is an open standard, specified at unifiedharnessprotocol.org, and HarnessRouter implements and complies with it: every harness on the platform runs behind that one contract. The open-source Community Edition server runs the same contract on your own infrastructure, so the complete path works hosted or on infrastructure you operate.
Because every harness runs behind the same contract, you can put real numbers on the choice before committing traffic. In the published same-task benchmark, eight configurations ran the same task, five runs each on identical input; across them, cost per task varied by approximately 475 times and p95 end-to-end latency by more than 3 times. Results vary by task, and that is the reason to measure yours.
The boundaries
The two paths compose
There are cases where the primitive path is the right call: you are building your own agent runtime end to end, you need deep control over isolation technology, snapshots, or GPU-backed execution, or your workload is custom code execution rather than complete agent tasks. In those cases you are building a runtime, not adopting a backend, and primitives are what that project needs.
The two paths also compose: sandbox infrastructure can sit underneath a complete backend, and a team can adopt the complete path for product features while keeping a primitive stack for specialized workloads. The decision framework in this article is per layer and per workload, not a single allegiance.
The checklist
A self-check before you commit an architecture
Before settling on either path, answer these against your roadmap:
- Which layers of the backend will your first production feature exercise in its first week?
- Who owns sessions, streaming, and file intake on your current plan, and what does that cost in engineering time?
- If your chosen harness is the wrong fit for a task class, what does switching cost you?
- Can you measure success, latency, and cost per task today, across more than one harness and model configuration?
- Which layers are differentiating for your product, and which are undifferentiated plumbing?
The conclusion
Complete is the requirement; the path is the choice
The seven jobs above are the whole requirement, and a production feature reaches them sooner than most roadmaps expect. The sandbox-first mental model is not wrong about isolation; it is early. Isolation is the first layer the backend needs, and the six around it decide whether the feature survives contact with users.
Decide layer by layer. Build the layers that differentiate your product, and adopt the ones that do not. If most of the list above is plumbing for you, the complete path exists, its contract is open, and it is ready to take a real task today.
FAQ
Backend for AI agents FAQ
What is a backend for AI agents?
It is the server-side system that turns a user's request into finished agent work. A complete one runs a harness in isolation, tracks tasks and sessions, streams progress, moves files in and artifacts out, and handles routing, retries, and metering across harnesses. Isolation alone is one of its seven layers.
Is a backend for AI agents the same as a BaaS for AI apps?
No. A BaaS gives an application databases, auth, and storage, and some now target apps that AI builds or operates. A backend for AI agents, as used here, is the runtime layer that executes agent work: harness, sandbox, sessions, streaming, artifacts, routing. A product often needs both, and they compose.
Do I need a sandbox for AI agents?
Yes. Agents act on real systems while they work, and that activity should stay isolated from production. The practical question is whether you buy isolation as a separate primitive and build the remaining layers yourself, or adopt a complete agent backend where sandboxed execution comes bundled with sessions, streaming, and artifacts.
What is a managed agent API?
A managed agent API is a hosted backend for AI agents behind one contract: you send tasks over an API, and the service runs complete harnesses, manages sessions and isolation, streams progress, and returns artifacts. It replaces assembling a runtime from primitives with a single integration.
Run your agent backend through one API
Turn the conclusion into the first task: create an API key, send a real task, and watch the stream and artifacts cover the layers this article described.
Send your first task