Trace Formats & Ingestion
Input format
faultgate reads an OTLP/JSON export that uses the OpenTelemetry GenAI semantic conventions. One trace is one agent run:
| Span | Attributes read |
|---|---|
root invoke_agent |
gen_ai.operation.name = invoke_agent, gen_ai.agent.name, gen_ai.input.messages (the user prompt), gen_ai.output.messages (the final answer), optional faultgate.termination (why the run stopped, e.g. step cap) |
child execute_tool |
gen_ai.operation.name = execute_tool, gen_ai.tool.name, gen_ai.tool.call.arguments, gen_ai.tool.call.result |
Set gen_ai.operation.name. A span without it is still read as a tool call when its name is execute_tool or execute_tool <tool> (the semconv span name), and precheck warns that it was recognised by name only.
Legacy gen_ai.prompt and gen_ai.completion are accepted as a fallback. If your exporter records structure but not content, faultgate warns you. examples/traces.json is a complete reference file.
Other trace formats
faultgate also reads the exports of common agent-observability tools and maps them onto the same attributes, so every check, policy and judge works unchanged. The format is detected automatically. To force one, use --format.
--format |
Export | Steps come from |
|---|---|---|
otlp |
OTLP/JSON, GenAI semantic conventions | execute_tool spans and tool calls inside chat messages |
openinference |
OTLP with openinference.span.kind (Arize Phoenix) |
TOOL spans: tool.name, tool.parameters or input.value, output.value |
langfuse |
trace objects with observations (e.g. GET /api/public/traces/{id}) |
TOOL observations; level: ERROR + statusMessage become span errors |
langsmith |
a list of runs | run_type: tool runs; error becomes a tool error |
braintrust |
a list of logged spans | span_attributes.type: tool spans |
--format accepts auto (the default), otlp, openinference, langfuse, langsmith or braintrust. It works on both check and baseline.
Token usage, finish reasons (context_overflow), errors (unrecovered_tool_error), the user id (enduser.id, for identity policies) and metadata.scenario (the scenario key for bands) carry over wherever the format records them. examples/formats/ holds the same refund run in each format. tests/test_formats.py checks that all four normalise to identical steps.
Precheck
Validate an export before you gate on it:
$ faultgate precheck examples/formats/langfuse.json
precheck: OK · format langfuse · checked 1 of 1 traces · 2 tool steps read · 0 error(s), 0 warning(s)It reports the detected format and how many tool steps the checks will read, then per trace: duplicate span ids, missing or backwards timestamps, tool calls without a name (errors, exit 2), and missing roots, spans whose parent is not in the export, runs with no prompt, answer or content, tool spans recognised only by their span name, spans that carry tool attributes but are not read as steps, and exports that mix native execute_tool spans with OpenInference LLM spans (warnings, exit 0). Compare the tool-step count with what your agent actually ran: a structurally valid export whose tools are not read passes every check by default. It inspects the first 5 traces by default (--sample N, 0 for all). On the 30 real benchmark traces from make demo-real it passes with warnings: those exports omit the root span.