CI & GitHub Action Integration

In CI

Add the action to a workflow. It runs the gate on your traces:

- uses: Faultline-lab/faultgate@main
  with:
    traces: traces/candidate.json      # exported by your agent's OTel pipeline
    band: traces/band.json             # from `faultgate baseline`, committed
    golden: traces/golden.json         # optional
    policy: traces/policy.json         # optional

The job exits 1 on gate FAIL, so the job fails. PASS and INCONCLUSIVE both pass. INCONCLUSIVE prints how many more scenarios to run. Every run also does the following:

• Writes the Markdown summary to the job summary. On pull requests, it posts one PR comment, updated in place on later pushes and found by its <!-- faultgate --> marker. The comment holds the verdict, gate numbers and p-value, fired checks, the policy hard block with each broken rule, failing traces with the step and reason, the latency, token and cost change against the baseline, and the exact command to reproduce the result locally. Give the job permissions: pull-requests: write, or set comment: "false".
• Uploads an audit artifact named faultgate-audit. It contains verdict.json (verdict, exit code, faultgate version, UTC time, every input file with its SHA-256, and the gate numbers), failing_traces.json (each failing trace and every reason), and repro.sh. No prompts, answers or tool payloads are copied into it, and reasons are masked by redact.
• Exposes the verdict as the step output verdict.

Other inputs: format, redact, checks, judge, report, audit, version. version takes a version (0.3.1), a specifier (>=0.3), a full requirement (faultgate==0.3.1) or any pip spec (a path, git+https://...); a bare version, v-prefixed version or specifier installs that faultgate version, and the default is the latest release. The same files come from the CLI with --markdown summary.md --audit audit/.

The action installs nothing heavier than pydantic. faultgate’s own CI runs it against this repo’s examples and asserts that the GPT candidate fails and the GLM baseline passes, with matching verdict outputs and audit files.

For the commands behind the action, see the CLI Reference. For how the verdict is decided, see Gate & Statistics.