OpenAI-Compatible Providers

AI Cost Firewall v0.4.2 supports practical OpenAI-compatible chat and embedding endpoints while keeping the flat configuration model.

v0.4.2 compatibility scope

v0.4.2 focuses on OpenAI-compatible APIs only:

• chat requests use the OpenAI-compatible /v1/chat/completions shape
• embeddings use the OpenAI-compatible /v1/embeddings shape when semantic cache is enabled
• provider root URLs and /v1 base paths are supported
• full endpoint paths should not be configured as base URLs

v0.4.2 does not add native provider-specific API integrations or provider-specific configuration blocks. Use the existing flat directives for both cloud and local OpenAI-compatible providers.

Supported patterns

Provider	Typical role
OpenAI	Chat and embeddings
Ollama	Local chat and embeddings
LM Studio	Local desktop inference
vLLM	Self-hosted GPU inference
LiteLLM	Aggregation/proxy layer
OpenRouter	Multi-provider upstream

upstream_provider openai_compatible;
upstream_base_url <base-url>;
upstream_api_key <key-or-placeholder>;

embedding_provider openai_compatible;
embedding_base_url <base-url>;
embedding_api_key <key-or-placeholder>;

The base URL may be either the provider root URL or its /v1 base path:

https://api.openai.com
https://api.openai.com/v1
http://ollama:11434
http://ollama:11434/v1
http://lmstudio:1234/v1
http://vllm:8000/v1
http://litellm:4000/v1

Do not configure the full endpoint path:

# Wrong
upstream_base_url http://ollama:11434/v1/chat/completions;

# Correct
upstream_base_url http://ollama:11434/v1;

For local providers without authentication, use a placeholder key:

upstream_api_key dummy;
embedding_api_key dummy;

Accepted placeholder values are dummy, none, null, and -.

Deployment examples

Runnable examples are available under:

deploy/examples/

Recommended examples:

openai-cloud/
local-ollama/
hybrid-openai-local-embeddings/
openrouter/
local-full-stack/

Runtime compatibility check

Use /version to confirm the running release and compatibility assumptions:

curl -s http://localhost:8080/version

The response includes supported_api_style and provider_specific_config_blocks so pilot operators can verify that the deployment is using the intended v0.4.2 compatibility model.

Guard orchestration and provider compatibility

VCAL Security Guard and VCAL Privacy Guard operate at the AI Firewall layer and are independent of the selected OpenAI-compatible upstream provider.

Provider compatibility still matters for:

• OpenAI-compatible chat request and response shape;
• model naming;
• streaming behavior;
• tool/function response formats;
• embedding endpoint behavior when semantic cache is enabled.

AI Firewall v0.4.2 supports non-streaming chat completions only. Requests with stream=true are rejected with HTTP 422 before cache, guard, or upstream processing.

The current guard modules inspect text content only. Non-text content such as images, audio, video, and binary payloads is preserved where possible but is not scanned, anonymized, or classified by AI Firewall guard modules.