In modern AI engineering, benchmark evaluations are frequently reported as single-pass success rates. An agent attempts an evaluation suite once, achieves an 85% pass rate on a specific split, and the result is published as an indicator of autonomous capability. However, when deployed into production pipelines with multi-step executions, that same system encounters unexplained breakdowns, stalled states, and catastrophic regressions.
The fundamental flaw lies in evaluating stochastic systems through single trials. A single successful run tells you almost nothing about operational stability.
01 / The illusion of the single pass
Autonomous agents execute non-deterministic reasoning loops over LLM calls, tool interactions, and environment feedback. When you evaluate an agent on a task once, the binary outcome—pass or fail—is a single sample drawn from an underlying probability distribution. If an agent has a true 70% probability of successfully completing a task, five independent runs will exhibit variance that single-pass benchmarks completely mask.
More critically, production workflows rarely consist of a single isolated invocation. Real agentic workloads require multi-step chains where an agent must execute reliably across repeated invocations without corrupting downstream state. If an agent has an 80% success probability per run, its probability of succeeding across five consecutive runs without failure drops to roughly 32.8%.
Without evaluating multi-trial consistency, teams mistake favorable variance for production readiness.
02 / Measuring reliability via pass^k
To measure true operational resilience, the FAULTLINE evaluation framework evaluates agent reliability across repeated trials using the pass^k metric, measuring joint success across k consecutive runs on identical tasks.
In our first publication on agent reliability, we conducted an empirical evaluation across 1,350 graded execution runs spanning diverse task categories. Rather than treating success as a static score, we measured how performance decays as the required consistency depth increases from k=1 to k=5.
A critical question in multi-trial evaluation is whether repeated trials exhibit coupling—such as cache leakage or progressive state corruption—or whether they behave as independent Bernoulli trials. Our findings confirmed trial independence within 2.5 percentage points of the binomial baseline across all evaluated runs. This empirical confirmation is vital: it establishes that pass^k decay is not an artifact of test interference, but an inherent characteristic of the agent’s stochastic policy.
When independence holds, evaluating pass^k provides an unforgiving lens. Models that appear competitive at k=1 rapidly diverge at k=3 and k=5, separating brittle architectures from truly stable reasoning engines.
03 / Tolerance bands over fixed thresholds
Standard evaluation leaderboards rely on point estimates, presenting single percentages that conceal confidence margins. When evaluating autonomous agents, point estimates create a false sense of precision.
Instead of fixed numerical thresholds, rigorous evaluation requires tolerance bands. In the FAULTLINE evaluation protocol, every pass^k measurement is bounded by empirically calibrated confidence bands derived from the 1,350 graded runs. When an agent’s multi-trial performance drops outside these bands, the degradation is flagged as an architectural instability rather than statistical noise.
Tolerance bands also allow teams to differentiate between minor variance and systematic divergence. If an agent maintains consistent performance within the calibrated band across repeated iterations, the policy is stable under perturbation. If the trajectory falls below the band, the failure indicates non-convergent planning or tool hallucination.
04 / Systematic failure taxonomy
Measuring pass^k decay identifies that an agent is failing, but engineering reliable systems requires knowing why it failed. To ground the quantitative results in mechanistic reality, we analyzed 200 coded execution traces from failed evaluation runs.
The resulting failure taxonomy identified four recurring failure modes:
- Context exhaustion: The agent exhausts context windows with unpruned tool outputs, causing catastrophic loss of the initial system prompt and constraints.
- Loop stagnation: Repetitive invocations of identical tools with minor parameter oscillations when encountering unexpected error messages.
- Premature termination: Signalling completion before verifying post-conditions or state mutations against task requirements.
- Tool contract violations: Emitting malformed tool arguments, unescaped JSON, or hallucinated method signatures that crash MCP handlers.
A single run might evade all four failure modes purely by chance. Over k=5 repeated runs, latent architectural bugs become almost impossible to conceal. Reliable agents are not built by celebrating lucky runs; they are built by measuring the steepness of the pass^k curve and hardening systems against the failures it exposes.