You pay for the agents you gate, not the tests you run. Every CI run, retry and replay is included.
Annual commitment: $1,500/mo per agent billed annually ($18,000/yr) vs $2,000/mo month-to-month
For individual devs and evaluation.
faultgate 0.3.0 + mcp-integrityFor teams shipping their first agent to production.
Billed annually ($18,000/yr per agent) · or $2,000 monthly
The managed layer on top of the free core.
Everything in Open Source, plus:
For orgs with multi-agent fleets and strict compliance.
A "gated agent" is one production agent with its own system prompt, tools, and business job (e.g. a billing-support agent).
2 agents × $1,500/mo billed annually ($36,000/yr)
Same price whether you run CI 10 times a month or 10,000. No per-trace or per-seat surprise.
| Capability | Open Source | Readiness Review | FAULTLINE Gate | Enterprise |
|---|---|---|---|---|
| Checks and rules | ||||
| Deterministic rule checks (6 rules)search_loop, abstention, policy, repeated_call, unrecovered_tool_error, context_overflow | Included | Included | Included | Included |
| Golden output checks (2 checks)wrong_answer, truncation | Included | Included | Included | Included |
| Model judge checkswrong_direction via local Laya or litellm API (BYOK) | Included | Included | Included | Included |
| Business rules in policy.jsonlimits (numeric min/max), identity (authenticated user), requires (tool prerequisites) | Included | Included | Included | Included |
| Policy violation as hard blockFails the gate immediately on any policy failure regardless of clean rate | Included | Included | Included | Included |
| Release gate | ||||
| Noise band & baseline calibrationfaultgate baseline measures run-to-run noise from 3+ runs | Included | Included | Included | Included |
| PASS / FAIL / INCONCLUSIVE verdictSignificance test (α = 0.05) vs baseline mean instead of coin-flip thresholds | Included | Included | Included | Included |
| Required sample size calculationCalculates additional scenarios needed at power 0.80 on inconclusive runs | Included | Included | Included | Included |
| CI exit codes & automationExit 0 (PASS, INCONCLUSIVE), 1 (FAIL), 2 (error) in any CI provider | Included | Included | Included | Included |
| PR comment & audit packGitHub Action job summary, in-place PR comment, verdict.json, repro.sh | Included | Included | Included | Included |
| Data and integrations | ||||
| Vendor trace importReads OTLP, Langfuse, LangSmith, Braintrust, Arize Phoenix (OpenInference) | Included | Included | Included | Included |
| Trace export precheckfaultgate precheck validates export structure before gating | Included | Included | Included | Included |
| PII and secrets redactionbuiltin (stdlib regex masks emails, cards, JWTs) or Presidio | Included | Included | Included | Included |
| Local execution (traces stay local)Runs in your CI runner; zero trace telemetry uploaded | Included | Included | Included | Included |
| Managed (Gate) | ||||
| Hosted verdict history & release ledgerPersistent audit trail across every PR and release | – | – | Design partner | Design partner |
| Domain rule packs maintainedPolicy packs written, tuned and kept current for your domain | – | – | Design partner | Design partner |
| Model-swap re-baseliningRe-calibrates noise bands when a model is swapped or retired | – | – | Design partner | Design partner |
| Alerts on FAIL / INCONCLUSIVEInstant notifications on regression or inconclusive test outcomes | – | – | Design partner | Design partner |
| Priority engineering supportDedicated channel and direct access to FAULTLINE engineers | – | – | Design partner | Design partner |
| Support | ||||
| Community support & discussionsPublic GitHub issues and documentation | Included | Included | Included | Included |
| 5-business-day readiness auditComplete review, policy writing and verdict delivery | – | Included | – | – |
| Self-hosted control planeDedicated control plane behind company VPC boundaries | – | – | – | Design partner |
| SSO / SAML integrationEnterprise identity and access management | – | – | – | Design partner |
| Enterprise SLAContractual uptime guarantees and vendor reviews | – | – | – | Included |
Everything marked Included for Open Source ships today in faultgate 0.3.0 (MIT). Items marked Design partner are on the roadmap: we build them with our first design partners, and they are not available yet.
When vendors charge per trace or per GB, engineering teams throttle test runs, reduce sample counts, or skip evaluating pull requests to stay under quota. That is the exact opposite of what reliability requires. Release gates should encourage continuous testing.
A flaky agent damages your business whether it ran 50 times in testing or 50,000 times in production. The cost to your business is the failure of that specific autonomous role. We price per gated agent so your release verification is exhaustive without financial friction.
A flat monthly rate per production agent. No surprise bills after an automated retry storm, test replay loop, or multi-hop stress test. You know exactly what your budget is on day one.
Observability platforms like LangSmith, Braintrust, and Confident AI charge by the trace, seat, or gigabyte. Several of them support repeated trials and custom code checks. Our difference is the combination: a release verdict rather than an observability dashboard, local-first execution where traces never leave your machine, deterministic checks before model calls, and flat per-agent pricing. See the full comparison →
One production agent with its own system prompt, tools, and business job (for example, a billing-support agent or a code-review agent). Sub-tasks, re-tries, and helper tool calls executed within that agent loop do not count as separate agents.
Metered pricing creates bad incentives: teams throttle test runs, sample fewer traces, or avoid re-running failing evaluations to stay under quota. That is the opposite of good engineering. We want you running CI as often as you commit code.
No. faultgate runs locally in your CI environment or terminal. Traces are parsed in memory, checks run on your machines, and candidate traces never leave your network. If you configure a model judge, calls go directly to your configured provider using your own API keys.
In faultgate 0.3.0, when a candidate run lands between the PASS and FAIL thresholds, the gate runs a one-sided significance test against the baseline mean. If the drop is not statistically significant (α = 0.05), it returns an INCONCLUSIVE verdict and calculates the exact sample size needed (at power 0.80) to distinguish regression from noise—for example, "needs 89 scenarios to tell from noise · run 59 more". This ships free in the open-source CLI and GitHub Action.
You pay the listed Gate price ($1,500 / agent / mo annual or $2,000 monthly) while we build the managed layer with you. The free core (faultgate 0.3.0) is complete on its own; design partners get direct access to our engineers to shape hosted verdict history, domain rule packs, automated re-baselining, and alerts before public self-serve availability.
The managed layer: hosted verdict history and release ledgers across every PR, rule packs written and maintained for your specific domain, automatic re-baselining when models are swapped or retired, instant alerts on FAIL or INCONCLUSIVE, and priority engineering support. If you only need to run the gate in your CI with your own policy files, you can self-host everything else for free forever.
Yes. faultgate 0.3.0 and mcp-integrity are open source under the MIT licence. You can install them with pip, run them in GitHub Actions, and use all deterministic rule checks, golden checks, business rules, and tolerance bands for $0 forever.
We write your policy.json business rules, import your traces from your current tool (OTLP, Langfuse, LangSmith, Braintrust, Phoenix), measure your noise band with 3+ baseline runs, and hand over the verdict + audit pack in 5 business days. Design partners receive this for $4,000.
Install faultgate locally in seconds or speak with us about gating your production agent fleet.
pip install faultgate