Skip to main content

Evidence Events

AI Cost Firewall emits structured evidence events using:

schema: vcal.evidence.event
schema_version: 1.1

Evidence events provide traceability across validation, guard orchestration, cache activity, upstream activity, and final request outcomes. In v0.8.0 Evaluation Mode, additive attributes also distinguish a hypothetical cache decision from the action actually applied to live traffic.

Enable evidence logging​

Set:

RUST_LOG=info,vcal_evidence=info

In Docker Compose:

environment:
  RUST_LOG: "info,vcal_evidence=info"

Lifecycle invariant​

Every trace that emits request.received ends with exactly one terminal event:

request.completed

or:

request.failed

A successful non-streaming upstream request can include:

request.received
cache.lookup.bypassed
upstream.request.sent
upstream.response.received
request.completed

A successful controlled streaming request can additionally include:

upstream.stream.completed

Provider-stream failures emit:

upstream.stream.failed

Stream-failure evidence records committed_to_client=false when the commit barrier prevented partial generated content from reaching the client. Idle and absolute-generation timeout failures use specific error classes such as stream_idle_timeout and stream_generation_timeout.

An error_json Security Guard block can include request.received followed by request.failed, with decision action block and the Security Guard rule_id. In completion response mode, the block decision is still represented in guard evidence even though the client receives a safe assistant completion.

Evaluation Mode attributes​

Evaluation-aware cache events can include:

enforcement_mode
decision
would_action
applied_action

Example shadow exact hit:

{
  "enforcement_mode": "observe",
  "decision": "exact_hit",
  "would_action": "serve_exact_cache",
  "applied_action": "call_upstream"
}

Example observed miss:

{
  "enforcement_mode": "observe",
  "decision": "miss",
  "would_action": "call_upstream",
  "applied_action": "call_upstream"
}

The evidence schema remains v1.1 because these are additive attributes. This separation is important for Audit/assessment workflows: a would-have action must not be interpreted as an action actually applied to the customer request.

Guard event names​

AI Cost Firewall uses explicit guard event names so request-side and response-side decisions can be queried without interpreting free-form messages.

Security Guard events:

guard.security.request.allow
guard.security.request.warn
guard.security.request.block
guard.security.request.failed
guard.security.response.allow
guard.security.response.block
guard.security.response.failed

Privacy Guard events:

guard.privacy.request.detected
guard.privacy.request.anonymized
guard.privacy.request.redacted
guard.privacy.request.failed
guard.privacy.response.restored
guard.privacy.response.restore_failed

When Security Guard returns a rule identifier, rule_id is preserved in the evidence event. If a response is blocked, Privacy Guard restore is skipped and the skip is reflected in orchestration metrics.

Buffered delivery to VCAL Audit​

When audit_enabled true; is configured, the same evidence records are placed on a bounded in-memory queue and delivered in batches to POST /v1/events/batch. Delivery is triggered by batch size or flush interval and uses configurable request timeout and exponential retry backoff.

The Audit path is asynchronous and does not normally block the client request. If the queue is full or a batch remains undeliverable after retry exhaustion, the affected evidence can be dropped and the failure is logged. The sender does not provide disk-backed replay.

Inspect events​

docker compose logs firewall | grep 'VCAL evidence event'

Filter lifecycle events:

docker compose logs firewall 2>&1   | grep -E 'request\.(received|completed|failed)'

Prometheus versus evidence​

Prometheus metrics provide aggregate operational visibility. Evidence events provide per-trace records.

AI Firewall Grafana dashboards show cost, cache, runtime, and orchestration health. VCAL Privacy Guard, VCAL Security Guard, and VCAL Usage Guard use their own dashboards for module-specific findings.

VCAL Audit is available as a separate evidence consumer for retention, search, trace timelines, lifecycle completeness, export, and SIEM/SOAR integration.

Compatibility​

Additive fields may be introduced within schema version 1.1. Breaking field changes require a new evidence schema version.