Skip to main content

Request Flow

v0.4.2 request flow​

AI Firewall first validates the request and rejects stream=true with HTTP 422 before cache, guard, or upstream processing.

In standalone mode, AI Firewall then follows the normal cache-first flow. When Security Guard and Privacy Guard are enabled, the request/response path becomes:

receive request
→ enforce request body and prompt-size limits
→ reject stream=true
→ normalize request
→ Security Guard request scan
→ Privacy Guard scan/anonymize/redact
→ check per-request cache bypass
→ exact cache lookup, if enabled
→ semantic cache lookup, if enabled
→ upstream request on miss or bypass
→ Security Guard response scan
→ Privacy Guard restore, if mapping exists
→ emit request.completed
→ response return

A request-side Security Guard block returns a structured HTTP 403 error before Privacy Guard, cache, or upstream processing. A response-side block returns HTTP 403 before Privacy Guard restore, and increments the restore-skipped metric. Failure paths emit request.failed; successful paths emit request.completed.

AI Cost Firewall processes requests through validation, cache lookup, upstream forwarding, and cache storage.

Exact cache​

The firewall checks Redis / Valkey for an identical normalized request.

Semantic cache​

If exact cache misses, semantic cache can search Qdrant for similar prompts.

Semantic cache entries include:

  • inserted_at
  • expires_at

Expired entries are skipped during lookup and never reused.

A candidate is reusable only if:

similarity_score >= semantic_similarity_threshold
AND
expires_at > now
AND
cached response payload is valid

Expired entries are filtered before similarity ranking.

When:

semantic_cache_fail_open true;

runtime semantic lookup failures behave like cache misses and requests continue upstream normally.

Upstream request​

If no valid cache hit exists, the request is forwarded to the upstream OpenAI-compatible provider.

The chat provider and embedding provider may use separate OpenAI-compatible endpoints.

Cache storage​

The upstream response can be stored in Redis and Qdrant.

Evidence lifecycle​

AI Firewall emits structured vcal.evidence.event schema v1.1 records through application logs.

Every trace that emits request.received ends with exactly one terminal event:

request.completed

or:

request.failed

The same trace_id correlates request validation, explicit guard.security.* and guard.privacy.* events, cache activity, upstream activity, and the terminal outcome. Security Guard rule_id values are preserved when available.

Buffered Audit delivery​

AI Firewall v0.4.2 can route structured evidence to a buffered HTTP sink:

request processing
  -> evidence event
  -> bounded in-memory queue
  -> batch by size or flush interval
  -> POST /v1/events/batch
  -> VCAL Audit

The Audit path is asynchronous. Temporary Audit failures do not normally fail the client request. Retry exhaustion can result in evidence loss because the current sender queue is memory-backed.