The world’s first unified interface for agent harnessesOpen Source4.4kPowered by UHP

InsightsNov 20, 202613 min read

AI Agent Reliability: How to Measure and Improve It

AI agent reliability means repeatable success, bounded failures and cheap recovery. Learn the metrics, failure modes and controls that make agents dependable in production.

Repeated task trials with a fault contained at one gate.
Repeatable outcomes and bounded failures are separate parts of reliability.

AI agent reliability is the probability that an agent completes the same task correctly across repeated runs, under perturbed conditions, with failures that stay small and recoverable. It is a property of the whole system (harness, model, tools, memory and sandbox), not of the model alone, which is why teams that want it in production treat it as an engineering discipline: repeated-run testing, trajectory capture, failure isolation and per-task cost tracking. HarnessRouter exposes supported agent harnesses behind one API, so the same task can be run and compared across harness and model combinations instead of being tied to a single configuration.

TL;DR

  • Reliability is repeated success under varied conditions with bounded failure severity, not a single passing run.
  • Capability gains do not automatically produce reliability gains: one evaluation of 15 models found only small reliability improvements as capability rose.
  • In the cited 150-incident corpus, most failures were classified as tool, orchestration or integration failures, so harness controls deserve explicit testing alongside the model.
  • Start with a small suite: repeated runs, trajectory capture, fault injection, quality gates and post-deployment monitoring.
  • Track cost per successful task, not cost per token, so reliability and spend are judged together.

What reliability means for an agent

A demo succeeds when an agent finishes a task once. A production system succeeds when it finishes that task on the thousandth run, after a tool changed its response format, with a longer context than the one you tested. These are different properties, and the gap between them is where most agent projects stall.

The research paper Towards a Science of AI Agent Reliability argues that single-metric evaluations omit four dimensions: consistency across runs, robustness to perturbations, predictability, and bounded error severity. Each maps to a practical question:

DimensionQuestion to askExample check
ConsistencyDoes the agent give an acceptable result every time it is run on the same task?Run each task 10 to 30 times and report the pass rate, not the best run
RobustnessDoes it hold when inputs or conditions change slightly?Rephrase the instruction, reorder fields, extend the context
PredictabilityCan you anticipate when it will fail?Compare stated confidence or escalation behavior with actual outcomes
Bounded severityWhen it fails, how much damage does it do?Classify failures: harmless retry, wrong output, destructive action

The fourth dimension is the one most teams skip. Two agents with an identical 95% success rate are not equally reliable if one fails by returning an error and the other fails by deleting the wrong file.

Why a better model is not the whole answer

The usual reaction to an agent failure is to swap in a larger model. The evidence for that being sufficient is weak. The same paper evaluated 15 models across two benchmarks and reported that capability gains yielded only small reliability improvements. A model that is more capable at its best can still be inconsistent from run to run.

That shifts attention to the surrounding system: how the harness structures the loop, how tools are described and retried, how state persists between steps, and what the agent is allowed to touch. These are design choices, and they can be changed without waiting for a new model release.

Tools, saved state and output checks surround a model, with a fault stopped at the tool gate.
Reliability depends on the model and on tool, state and validation boundaries. The illustration does not imply that changing models cannot help.

Where agents actually fail

A useful taxonomy comes from an analysis of 150 production incident reports, which organizes 23 failure modes into five categories: retrieval, generation, tool, orchestration and integration (Compound AI System Reliability). The examples the paper names include stale retrieval, hallucinations triggered by noisy context, tool timeouts, schema drift, retry loops and type errors at component boundaries.

In practice, this means a reliability review should walk the agent's path and ask what happens at each boundary:

  1. Retrieval: is the data fresh, and does irrelevant context get filtered before it reaches the model?
  2. Generation: can the output be validated against a schema or a rule before it is used?
  3. Tool calls: what happens on a timeout, a changed response shape or a partial result?
  4. Orchestration: can a retry loop run indefinitely, and who stops it?
  5. Integration: do values passed between components keep their types and meaning?

Tool hazards deserve their own tests

Tools rarely fail cleanly. A benchmark paper, Beyond Function Calling, tests recoverable hazards: specification drift, invocation errors, execution failures, output drift and conflicts between sources. Its finding is that agents that do well with reliable tools often fail under these hazards, and that the failures come more from weak hazard diagnosis and ineffective recovery than from the number of tool calls or the inference budget.

The implication is direct. Giving an agent more tokens or more attempts does not repair a failure it cannot diagnose. Recovery logic, meaning detecting what went wrong and choosing a different action, is what to test.

How to test for reliability

The Singapore Infocomm Media Development Authority's Starter Kit for Testing Agentic Systems (version 1.0, October 2026) is a practical baseline. It recommends realistic, longer-horizon tasks, multiple runs to account for variability, representative task variations, different model and harness combinations, long-context tests, and recording the configuration under test: model version, prompts, settings, tools and permissions.

A small first reliability suite

A team does not need a large evaluation program to begin. A workable first suite has six parts:

  1. Representative tasks. Pick 20 to 50 tasks drawn from real usage, including some that are long and multi-step.
  2. Repeated runs. Run each task several times and report the pass rate and its spread. A task that passes 6 of 10 times is a reliability problem even if its best run is perfect.
  3. Reset between trials. Start each run from the same clean state so one trial's side effects do not contaminate the next.
  4. Boundary probes. The paper Reliability Engineering for AI Systems recommends testing complete task trajectories under representative conditions and deliberately probing boundaries: partial tool failures, ambiguous instructions, malicious retrieved content and high-risk actions.
  5. Configuration log. Record model version, prompt, tool set and permissions with every result so a regression can be traced.
  6. Retest schedule. Rerun the suite when the model, harness or tools change, and periodically even when they do not.

The same paper frames pre-deployment testing as initial evidence within a specified operating envelope, which field monitoring then has to confirm keeps holding. Testing defines what you know on day one; monitoring tells you whether it is still true.

Why comparing harnesses matters

Because reliability depends on the harness as well as the model, the useful experiment is a matrix: the same tasks across several harness and model pairings. Doing that normally means writing an integration for each harness. With HarnessRouter, harnesses such as Codex, Claude Code, Hermes, DeepSeek and Pi are reachable through one API, and the platform includes benchmarking tools, so the matrix becomes a configuration change. For the method behind this kind of comparison, see our guide to testing models on your own workload.

Metrics worth tracking

A reliability dashboard needs fewer numbers than most teams collect, provided they are the right ones:

MetricWhat it tells you
Task success rate across repeated runsConsistency, reported as a distribution rather than a best case
Verified outcome rateWhether the result was checked against ground truth, not just reported as done by the agent
Failure severity distributionShare of failures that are harmless, wrong-but-contained or destructive
Recovery rateShare of tool hazards the agent diagnosed and recovered from
Latency, median and tailWhether retries and recovery push response time past what the product can tolerate
Cost per successful taskTotal spend divided by tasks that passed verification

Cost per successful task is the metric that joins reliability to budget. An agent that costs half as much per attempt but succeeds half as often costs the same per success, and adds more failures for users to see. For the trace data behind these numbers, our piece on what to trace, measure and audit lists the events worth capturing.

Controls that improve reliability

Testing measures reliability; controls produce it. The experimental results reported in the compound-systems paper give a sense of which controls pay off. In controlled fault-injection experiments, circuit breakers reduced cascade depth from a mean of 3.8 components to 0.4, which the paper describes as an 89% reduction. In the same experiments, output-quality gates caught 73% of silent degradation incidents and component isolation reduced blast radius by 64%. These are results from one paper's experimental setup, so treat them as evidence that the controls work in kind, then measure the effect on your own workload.

Circuit breakers and bounded retries

A retry loop with no ceiling converts a transient tool failure into a runaway cost and a stalled session. Set a maximum retry count per tool, back off between attempts, and open a breaker that stops calls to a failing dependency so one bad component does not drag down the components downstream.

Output-quality gates

Silent degradation is the hardest failure to catch because nothing errors. A gate validates output against a schema, a rule or a second check before the result moves on. Put gates at component boundaries, where the compound-systems taxonomy locates many type and integration errors.

Isolation

Isolation limits blast radius in two ways. Component isolation stops a fault in one stage from propagating. Execution isolation, running the agent in a sandbox, limits what a wrong action can touch. HarnessRouter runs each task in an isolated sandbox, which bounds local file and process effects. Outbound API actions still need separate authorization, scoped credentials and idempotency controls.

Trajectory capture and monitoring

The IMDA starter kit recommends capturing trajectories: actions, tool calls, interactions, state changes and outcomes, followed by continuous monitoring and periodic retesting after deployment. Without trajectories, a failed run is a single line saying it failed. With them, you can see which step diverged and write a regression test for it.

Durable state

Long tasks fail midway. Session management and persistent memory let an agent resume instead of restarting, which can reduce repeated work and cost. Avoiding repeated external side effects also requires stable action identifiers, enforced idempotency or reconciliation. HarnessRouter includes agent session management and file handling, and offers Harness Memory for long-term agent memory.

Balancing reliability, latency and cost

Every control above adds something: a gate adds a check, a retry adds latency, a stronger model adds spend. The decision is easiest when each task is routed on its own requirements rather than the whole product sharing one configuration. A routine extraction task can run on a lean model with a strict output gate; a high-risk action can use a stronger model and a human approval step.

Three rules keep this honest:

  • Set a reliability threshold per task type before choosing a model, then pick the cheapest configuration that clears it.
  • Compare configurations on cost per successful task, using repeated runs.
  • Re-evaluate when models change, since a cheaper option may now clear the bar. Our method for matching models to tasks covers the selection process in detail.

HarnessRouter lets teams benchmark and configure compatible harness, model and module choices for each task, which makes that per-task routing a configuration decision rather than a rewrite. Its stated aim is to cut AI agent costs by over 90%.

Design reliability in before the workflow ships

Many reliability incidents trace back to decisions made before any code ran: which workflows an agent should own, what it may touch and who reviews its output. A guide on agentic AI before it enters workflows walks through the planning questions that shape those boundaries, and they pair well with the failure categories above. For the engineering side, our overview of AI agents in production covers architecture, controls and costs, and deploying AI agents covers the move from prototype to production.

Our perspective

We treat the agent harness as the layer where reliability is decided. Models change every few months; the loop that calls them, the tools they reach, the memory they read and the sandbox they run in determine whether a task finishes the same way twice.

That is why HarnessRouter makes every part of an agent system, from harnesses and models to skills, tools, plugins, memory and sandboxes, an interchangeable building block. When each part can be swapped independently, a reliability problem becomes a controlled experiment: change one block, rerun the suite, read the result.

It is also why we built on the open Unified Harness Protocol (UHP). An open protocol keeps the harness layer from becoming a lock-in point, so a team can move a workload to whichever harness and model clear its reliability bar at the lowest cost per successful task.

Run the comparison on your own tasks

HarnessRouter gives developers and product teams one API to run agent tasks across many harnesses, models and sandbox providers, with benchmarking tools, session management and an isolated sandbox for each task. The Free plan costs $0 per month, the Developer plan $20 per month, the Production plan $200 per month and the Scale plan $1,000 per month; Tracing is in beta and included with Scale; see the pricing page for current add-on terms. The BYOK routing fee is 0% for up to $25,000 in monthly model usage at list prices, with a 5% routing fee above that volume when you bring your own model API keys; agent work and memory remain separately billable. Teams that prefer to self-host can use the open-source Community Edition. Plan details are on the pricing page.

HarnessRouter

FAQ

What is the biggest problem with AI agents in production?

Inconsistency and weak recovery. Agents that succeed in a demo often fail when a tool times out, changes its response format or returns conflicting data, and research on tool hazards attributes those failures mainly to poor diagnosis and ineffective recovery. Fixing it means testing recovery paths, not only the happy path.

How many times should I run each test?

Enough to see the spread in outcomes, which for most teams means repeating each task several times and reporting the pass rate. The IMDA starter kit recommends multiple runs because agent behavior varies. Increase the count for high-risk tasks, where a small failure rate still matters.

Is reliability the same as accuracy?

No. Accuracy describes how often an output is correct on average. Reliability adds consistency across runs, robustness to changed conditions, predictability and limits on how severe a failure can be, the four dimensions set out in the PMLR paper.

Which controls should I add first?

Start with bounded retries, an output-quality gate at the most critical boundary and trajectory capture. In one set of fault-injection experiments, circuit breakers reduced cascade depth and quality gates caught silent degradation. Add isolation for code execution, and authorization and idempotency controls for external actions.

Sources

→Next

Run the comparison on your own tasks

HarnessRouter gives developers and product teams one API to run agent tasks across many harnesses, models and sandbox providers, with benchmarking tools, session management and an isolated sandbox for each task.