- Home
- LLM Gateway Software
- OpenZiti LLM Gateway
On the recordRead 10 Oct 202620 facts
OpenZiti LLM Gateway
Charlotte, North Carolina, United States
On the record
- Anthropic support
- The gateway translates between OpenAI format and Anthropic’s Messages API.github.com · read 9 Oct 2026
- Client authentication
- Optional virtual API keys can be restricted to models using glob patterns; the health and metrics endpoints remain unauthenticated.github.com · read 10 Oct 2026
- Deployment
- The project describes deployment as one Go binary and one YAML file, with no database, message queue, or sidecar required.github.com · read 10 Oct 2026
- Integrations
- Documented backends include OpenAI, Anthropic, Ollama, vLLM, llama-server, and SGLang; clients use an OpenAI-compatible API.github.com · read 10 Oct 2026
- Intended users
- The README describes routing across local inference servers and remote providers, including GPU servers behind NAT and clients using OpenAI-compatible libraries.github.com · read 10 Oct 2026
- License
- The repository identifies the project as Apache 2.0 licensed.github.com · read 10 Oct 2026
- Load balancing
- It supports weighted round-robin across inference endpoints, health checks, and failover.github.com · read 10 Oct 2026
- Metrics
- When enabled, its Prometheus endpoint reports request, latency, token, routing, provider-error, and endpoint-health metrics.github.com · read 10 Oct 2026
- Network security
- The project describes using zrok over OpenZiti to connect gateway and backend across NAT or network boundaries without firewall rules or port forwarding.github.com · read 9 Oct 2026
- Observability
- When enabled, Prometheus-exported metrics include request counts, latency, token counts, routing decisions, provider errors, and endpoint health.github.com · read 9 Oct 2026
- Provider integrations
- The project documents routing to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang, and other OpenAI-compatible backends.github.com · read 9 Oct 2026
- Provider translation
- It translates between the OpenAI request format and Anthropic’s Messages API.github.com · read 10 Oct 2026
- Purpose
- It is an OpenAI-compatible API proxy that routes requests to OpenAI, Anthropic, and compatible backends such as Ollama and vLLM.github.com · read 10 Oct 2026
- Routing
- Optional semantic routing uses keyword heuristics, embedding similarity, and an LLM classifier to choose a model when the request omits one.github.com · read 9 Oct 2026
- Semantic routing
- An optional cascade of keyword heuristics, embedding similarity, and an LLM classifier can select a model when the request omits one.github.com · read 10 Oct 2026
- Streaming
- It supports server-sent event streaming for its providers.github.com · read 10 Oct 2026
- Tracing
- Optional tracing logs request metadata and truncated message content, with a configurable maximum content length.github.com · read 9 Oct 2026
- Zero-trust networking
- With zrok over OpenZiti, the gateway can expose itself or reach backends without firewall rules or port forwarding.github.com · read 10 Oct 2026
