Skip to main content

Runtime Overview

AI Cost Firewall supports:

  • health and readiness endpoints
  • graceful shutdown
  • request draining
  • upstream and embedding timeout handling
  • hot reload through SIGHUP
  • runtime metrics
  • semantic cache fail-open behavior
  • OpenAI-compatible provider diagnostics
  • embedding provider timeout visibility
  • release and compatibility introspection through /version
  • optional Security Guard and Privacy Guard orchestration
  • configurable guard fail-open/fail-closed behavior

Startup dependencies​

Redis is required for exact caching.

Qdrant is required when:

semantic_cache_enabled true;

semantic_cache_fail_open applies to runtime semantic lookup failures only.

AI Cost Firewall validates runtime dependencies during startup and reload. v0.4.2 startup diagnostics are intended to make pilot deployment failures easier to identify before traffic is sent.

This includes:

  • loaded configuration summary
  • Redis connectivity
  • Qdrant connectivity
  • semantic cache configuration completeness
  • vector-size compatibility
  • OpenAI-compatible upstream and embedding provider configuration

During graceful shutdown:

  • readiness becomes unavailable
  • new requests are rejected
  • in-flight requests continue

AI Cost Firewall supports nginx-style configuration reload using:

SIGHUP

semantic_cache_fail_open affects runtime semantic lookup behavior only and does not bypass startup validation.

Version endpoint​

The /version endpoint reports the running release and compatibility model. For v0.4.2, it confirms that AI Cost Firewall is a pilot-ready OpenAI-compatible gateway and that provider-specific configuration blocks are intentionally not part of this release.

Guard runtime behavior​

AI Firewall v0.4.2 can orchestrate VCAL Security Guard and VCAL Privacy Guard.

Recommended full enterprise order:

Security Guard request scan
→ Privacy Guard scan/anonymize/redact
→ exact/semantic cache lookup or upstream LLM
→ Security Guard response scan
→ Privacy Guard restore

guard_fail_open controls what happens when an enabled guard is unavailable, times out, or returns an invalid response contract.

Recommended enterprise setting:

guard_fail_open false;

This fails closed instead of sending unscanned or unanonymized traffic to cache, upstream providers, or clients.

Streaming requests are rejected globally in v0.4.2, regardless of whether guards are enabled. The rejection occurs before cache, guard, or upstream processing.

Evidence lifecycle​

AI Firewall emits structured evidence events through application logs using schema vcal.evidence.event version 1.1.

For every trace that emits request.received, runtime processing emits exactly one terminal event:

  • request.completed for successful delivery;
  • request.failed for validation, guard, cache-fatal, or upstream failures.

Security Guard blocks use evidence action block and preserve the Security Guard rule_id when available.

v0.4.2 validation​

AI Firewall v0.4.2 was validated with VCAL Privacy Guard, VCAL Security Guard, and VCAL Audit. Validation covered guarded requests, buffered evidence delivery, trace reconstruction by trace_id, SQLite persistence, and successful SHA-256 record-chain verification.