Audit journal: verifying it and reporting from it

Published from the project repository

This page is generated from the project’s documentation. References to source files, tests and unpublished documents appear as plain text, and test names such as suite::test_name identify the evidence behind a claim. The source repository is not public yet, so that evidence is available on request.

Written 2026-09-03 for Phase B (WPs B.5, B.6, B.7) of the enterprise product plan (retired Enterprise repository, docs/enterprise-product-plan.md). Everything here runs on a copy of the journal on an operator’s machine; nothing needs the gateway or the network.

What is in the journal

MCP_AUDIT_JOURNAL_DIR/audit-journal.jsonl: one JSON object per line, with a contiguous seq from 1 and a kind:

kind	Written when	Carries
tool_call_intent	before a tool dispatches	principal, canonical action, decision (effect, rule, policy version), authorization (the generation the decision was taken under, with its policy and revocation-list versions), upstream identity
tool_call_outcome	after it completes	outcome, duration, every egress decision and what the transport did with it, and subprocesses: each program the call ran that reaches the network outside the egress chokepoint (today the git behind bb_clone), with the validated, credential-free URL it was handed and whether it succeeded, failed, or was still started when the call ended
egress_decision	the egress chokepoint denies or holds an upstream request	the refused action
authorization_generation_loaded / authorization_generation_changed / authorization_generation_rejected	the signed generation binding the policy and the revocation list is loaded, changed (both halves put in force together, after this record is durable), or refused (an unsealed document, a manifest that is not newer, a signature that does not verify)	the generation and its label, policy_version and revocation_version (and on a change the previous_* ones), counts, not_before, signature status, reason
policy_loaded / policy_changed / policy_rejected	a policy with no generation (local dry runs) is loaded, changed, or refused	version labels (bundle hashes), signature status, reason
revocation_loaded / revocation_changed / revocation_rejected	a revocation list with no generation (library embedders) is loaded, changed, or refused	counts, not_before, signature status
revoked_token_rejected	a validated token is refused by the revocation list	the principal, the reason (subject, token_id, not_before), the list version, and the generation
admin_mutation	an administrator’s mutation is authorized, before its effect	the principal, the operation (source), the target or version label
call_proposal / call_approval / call_rejection / call_consumption / call_application	a tool call a policy rule held is recorded for a second person, approved, rejected or found expired, consumed by the requester’s re-issue, or completed (MCP_CALL_APPROVALS=required, issue #41)	the principal, source (tool/<name>), version (the vendor), proposal (id, kind: tool_call, candidate_digest = the hold key, expires, a redacted summary; consuming_request on consumption), applied_seq and outcome on completion, reason on rejections. The held call’s own tool_call_intent names the proposal under approval (proposal, and consumed_seq on the executing re-issue)
admin_proposal / admin_approval / admin_rejection	a gated mutation is held for a second administrator, approved, or rejected / found expired (MCP_ADMIN_APPROVALS=required)	the principal, the operation, the version label, proposal (id, candidate_digest, expires; the proposal record also carries the exact document and signature), reason on rejections
resource_decision	before disclosing each GitHub issue in an enforcing scope	trusted-label policy decision and request context; no response labels or contents. Direct refusals classify the call as resource_denied; its egress record still reports the upstream GET. Denied list rows are omitted.
checkpoint	every N records, T seconds, and at clean close	the chain value over everything before it, signed

Never a token, never tool arguments, never response content (plan §3.3). Search expressions and caller-supplied values appear as digests.

Verifying a copy

mcp-devtools audit verify /path/to/journal-dir \
    --public-key "$AUDIT_PUBLIC_KEY" \
    --checkpoints /path/to/exported-checkpoints

--public-key is the public half printed by mcp-devtools audit keygen (repeat it for every key a journal spans). --checkpoints is a copy of MCP_AUDIT_CHECKPOINT_EXPORT_DIR. The verifier:

1. reads every line and requires seq to be the previous one plus one;
2. recomputes the SHA-256 chain (chain = H(chain ‖ line), seeded) and compares it to each checkpoint’s chain;
3. checks each checkpoint covers exactly the range since the previous one and that its signature verifies with the key it names;
4. compares every exported checkpoint to the journal’s, byte for byte, flags any export that lies beyond the journal’s last record — a truncated tail, which the chain alone cannot see — and flags every journal checkpoint the export directory lacks (a copy that failed or was removed; the gateway logs audit checkpoint export failed when it happens) and any sequence exported twice.

Exit code 0 means no problem was found; 1 means at least one was, and the report names each (--json for machines). What is not covered: records after the last checkpoint, until the next one is written (bounded by MCP_AUDIT_CHECKPOINT_RECORDS / MCP_AUDIT_CHECKPOINT_SECONDS, and sealed at every clean shutdown). A gateway that died mid-run leaves an unsealed tail; the report says how many records.

Activity export

mcp-devtools audit export activity /path/to/journal-dir \
    --since 2026-09-01T00:00:00Z --until 2026-10-01T00:00:00Z \
    --subject alice@acme.example --vendor grafana \
    --kind tool_call_intent --kind tool_call_outcome \
    --format csv --out activity.csv

One row per record (checkpoints excluded), flattened to fixed columns: sequence, timestamp, kind, request id, subject, tenant, groups, tool, vendor, environment, normalized action, resource type, resource scope, request risk, effect, rule id, policy version, reason, upstream identity label and authority, outcome, duration, egress counts (allowed / denied / attempted), the document version a control record is about, and — appended in the order they arrived, so a reader of an earlier export by column position keeps its columns — egress_held, which counts approval holds (allowed + denied + held reconciles to the retained request list), then the authorization generation with its revocation-list version: for a decision, the pair it was taken under; for a generation record, the one it is about. Attempted includes sends still in_flight, whose response was never recorded. Intent and outcome rows share a request_id; they are not merged, because the journal’s own granularity is the evidence. --format jsonl gives the same rows as JSON Lines. Every filter is optional; time bounds are RFC 3339 at any precision and are compared at full precision.

Activity rows include an optional subject_display, taken from the validated OIDC token’s preferred_username, then email. Names are trimmed, limited to 256 UTF-8 bytes, and omitted when blank, malformed, or containing control characters. The console shows this name with the stable subject underneath; CSV appends it as the last column and JSON Lines includes it as a field. Older journal records and stored activity rows have no display name.

This is display metadata recorded at the time of the call: names may change or collide. subject remains the identity key for policy, audit correlation, and report filtering. Display names never affect authorization or its cache.

The CSV is spreadsheet-safe: a cell that a spreadsheet would evaluate — one beginning with =, +, -, @, a tab, or a carriage return — is written quoted with a leading apostrophe ("'=HYPERLINK(...)"), which Excel, Sheets, and LibreOffice display as text. Subjects, groups, rule ids, and reasons all come from outside the gateway, so an export that is going to be opened must not let any of them run. A pipeline that parses the CSV should strip that apostrophe, or use --format jsonl, which is unchanged.

A journal that cannot be read to the end exports what it could and exits non-zero with the reason.

Access review

mcp-devtools audit export access-review \
    --policy policy.yaml --public-key "$MCP_POLICY_PUBLIC_KEY" \
    --groups groups.yaml --tenant acme --format csv --out access-review.csv

groups.yaml is group → members, exported from the identity provider:

SRE: [alice@acme.example, bob@acme.example]
Contractors: [carl@contractor.example]

Pass --scopes scopes.yaml to provide subject → context → scopes minted together:

alice@acme.example:
  browser: [mcp:tools]
  reporting-client: [reports:read]
bob@acme.example:
  restricted-client: []

JSON is also accepted. Context labels are operator-supplied identifiers, not credentials or authorization inputs. Scope sets are evaluated independently; they are never combined across clients or tokens. A known empty array matches no scope requirement. A missing subject, omitted snapshot, or empty context map means unknown scopes, never the default MCP_REQUIRED_SCOPE.

The review lists rules matching each subject’s groups and each supplied scope context. Subjects named directly in a rule or in the scope snapshot are included even when absent from groups. Rules requiring scopes remain visible as candidates when scopes are unknown; non-scope subject constraints still have to match. CSV appends context, scopes, scope_match, and required_scopes columns; JSONL contains the same fields. scope_match is unknown for a scope-dependent candidate and matched otherwise. Consumers must inspect scope_match: effect remains the rule’s allow/deny value even when the scope match is unknown. JSON scopes: null means unknown and [] means known empty; CSV encodes the scopes cell as the same JSON array or null. required_scopes contains the rule’s scope patterns (any one may match).

These are supplied snapshots, not a live identity-provider fetch. A matched rule is not a final access decision: resource constraints, deny precedence, inbound authentication requirements, and revocations still apply at request time. The existing report excludes rules requiring a verified principal kind; this snapshot does not supply kind evidence. Group membership is shared across a subject’s contexts, so use separate reviews if their tokens carry different groups.

Flow and budget columns (runtime security plan §8, §9)

After the inference columns the export appends owner_classes (the owner’s taint classes an intent was decided on), labels (the classes a result or completion was labelled), marked (the classes the outcome marked on its owner), and budget_ids, budget_states and budget_settlements, each ;-joined in the record’s order so the three budget columns line up; an entry without a settlement (an intent) leaves an empty item. A flow denial is an ordinary deny whose rule_id is the flow rule and whose reason names the class and the destination. Nothing in these columns is content: a class is an id from the bundle’s classes.

Data map (runtime security plan §9)

mcp-devtools audit report --data-map --tenant acme \
  --store sqlite:///var/lib/mcp-devtools/usage.db [--since …] [--until …] [--format csv|jsonl]

One row per (subject, vendor, tool, model, class) the tenant’s principals touched in the window, with the count and the first and last timestamp, read from the rollup store’s class rows (the file, or MCP_ROLLUP_STORE). It is the same report the admin API serves at POST /admin/reports/data-map and the console draws on its Data map page, both bound to the caller’s tenant; the CLI takes the tenant explicitly because a file has no caller. It is a rollup, not journal evidence: the window is bounded by MCP_ROLLUP_RETENTION_DAYS, and the marked column of the activity export is the record. See the usage runbook.

Trial metrics (plan §7)

mcp-devtools audit metrics /path/to/journal-dir [--since …] [--json]

§7 metric	Field
Time to provision (group add → first successful call)	per subject, time_to_first_allow_ms (first attempt → first success; the identity-provider half of the interval is not in the journal)
Time for revocation to take effect	per subject, revocation_window_ms (the authorization_generation_changed whose revocation version moved, or a standalone revocation_changed → first revoked_token_rejected) and deny_after_allow_ms (last allowed call → first policy denial, the group-removal proxy)
Share of requests covered by an explicit allow rule	explicit_allow_share (allowed_by_rule / calls)
Cross-environment access attempts blocked	cross_environment_attempts (denied calls by a subject allowed for the same vendor in another environment)
Time to produce a report	elapsed_ms on stderr of every export and metrics run

Counts are over tool_call_intent records; outcomes, egress denials, policy and revocation changes, and checkpoints are counted separately.

Approval-required decisions

A policy require_approval verdict is reported separately from allow and deny. Journal metrics include held globally and per subject; activity and access-review exports retain the literal effect require_approval. For admission holds, usage records use decision: held and outcome: approval_required. Rollup totals, groups, and timelines include a held count. Held calls do not contribute to denial counts, cross-environment denial attempts, or execution latency. They do remain in the denominator of explicit_allow_share (allowed_by_rule / calls) and rollup allow_share (allowed / rows): both measure the share of all attempts allowed, not just decided allow/deny attempts. One explicit allow and one hold therefore yield a 50% allow share with zero denials. Read the share alongside held; a burst of holds lowers the share without increasing denials. A held call that later executed under a consumed approval is counted as approved (globally and per subject) and in the subject’s allowed, never as held; it does not move explicit_allow_share.

Admission holds return isError: true, _meta.code: approval_required, and _meta.decision containing the effect, reason, rule id, and policy version. Text identifies the rule without assuming which approval backend is installed. Every other admission refusal also carries its outcome label in _meta.code.

An egress hold journals outcome: egress_approval_required, marks the blocked request’s dispatch as held, and increments egress_holds rather than egress_denials. Later requests blocked by that hold are also recorded, with outcome: egress_blocked_by_prior_hold and dispatch state blocked_by_hold; their decision fields still say what policy decided (allow and the matching rule), because the barrier is the transport’s doing, not the policy’s. They count as egress_blocked_by_hold in the metrics report and as egress_allowed in exports. Each request is evaluated independently: another hold or deny records its own matching rule, rather than inheriting the first hold’s rule. The call’s outcome record retains the admission decision unchanged.

Usage answers what happened to the call. An admitted call held at egress before any mutating send records decision: held, exactly like an admission hold, so rollups count it under held rather than as an allowed call that failed, and its wait never enters mean_duration_ms. Journal admission counts still say allow for it; held in the journal report counts admission holds only, and egress_holds explains the difference between the two reports.

When no mutating send began, the call outcome is egress_approval_required and the response has _meta.code: approval_required, retry_safe: true, and execution: not_started, or execution: reads_only when a request policy classified as a read went on the wire before the hold (a GET, or a reviewed read-only POST such as the Jira JQL search): nothing upstream moved, so the call is safe to re-issue once approved, but the caller is not told that nothing happened. Whether a send counts as mutating follows policy’s risk class for the request, not its HTTP method. If a mutating send began, including one with an unknown result, the call outcome and response code are partial_execution, and usage keeps decision: allow. The response preserves controller content, sets execution: partial_or_unknown and retry_safe: false, and warns against retrying the whole tool call. Earlier requests may already have taken effect; reconcile those effects before issuing another call. A protocol-level error raised by the controller after a hold is returned as that error, code and data intact, and classified protocol_error; the hold remains visible in the outcome record’s egress list. The client was never shown the hold, so its usage row is decision: allow with outcome: protocol_error: the usage decision follows from the one outcome classification, never from a separate look at the hold. Task and input-required responses retain their handles and opaque state, with the warning in _meta.message and retry_safe: false. Without a known send, their execution state is pending_or_unknown. Reconcile the existing continuation instead of issuing a duplicate tool call. Completed errors do not repeat an approval message already supplied by the controller. These indicators remain conservative even when the bounded egress log omits requests, including AWS sends whose futures are dropped. PR 1 creates no proposal and does not replay calls.

The denials-by-environment rollup includes only admission denials; held-only groups do not appear. Totals, general group reports, and timelines retain separate held counts. Egress barriers are execution outcomes on admitted calls. The audit service records the caller’s supplied outcome without reclassifying it.