- Home
- LLM Gateway Software
- Inference Gateway
On the recordRead 10 Oct 202618 facts
Inference Gateway
On the record
- Agent coordination
- Its Agent-to-Agent support lets agents discover capabilities, delegate tasks, and stream results within a conversation.docs.inference-gateway.com · read 10 Oct 2026
- Agent workflows
- The product supports Agent-to-Agent coordination and describes defining agents in YAML with a CLI that generates Go, Rust, or TypeScript projects.docs.inference-gateway.com · read 9 Oct 2026
- Authentication
- Authentication supports OIDC identity providers, with JWT validation against the issuer’s JWKS.docs.inference-gateway.com · read 10 Oct 2026
- Deployment
- The getting-started guide documents Docker, Docker Compose examples, and Kubernetes deployment.docs.inference-gateway.com · read 10 Oct 2026
- Desktop platforms
- The desktop app page lists Linux, macOS, and Windows assets for amd64 and arm64; it also says releases are not signed with Apple Developer or Windows code-signing certificates.docs.inference-gateway.com · read 9 Oct 2026
- Footprint
- The gateway is described as an approximately 10.8 MB static binary designed to scale horizontally with Kubernetes HPA.docs.inference-gateway.com · read 10 Oct 2026
- Kubernetes
- A Kubernetes Operator manages gateways, agents, MCP servers, and chat orchestrators declaratively as cluster resources.docs.inference-gateway.com · read 10 Oct 2026
- License
- The documentation states that Inference Gateway is released under the Apache-2.0 license.docs.inference-gateway.com · read 9 Oct 2026
- License and support
- The project is released under Apache-2.0, with GitHub Discussions for questions and an issue tracker for bugs and feature requests.docs.inference-gateway.com · read 10 Oct 2026
- MCP integration
- It can discover tools from connected MCP servers and execute tool calls server-side.docs.inference-gateway.com · read 10 Oct 2026
- Observability
- Built-in observability includes Prometheus metrics, OTLP tracing, structured JSON logs, and reference Grafana dashboards.docs.inference-gateway.com · read 10 Oct 2026
- Privacy
- The project states that it has no analytics or telemetry phoning home and can be self-hosted on-premises, in the cloud, or air-gapped.docs.inference-gateway.com · read 10 Oct 2026
- Product
- Inference Gateway is an open-source, cloud-native proxy that presents supported hosted and local LLM providers through one OpenAI-compatible API.docs.inference-gateway.com · read 9 Oct 2026
- Provider coverage
- The documentation lists providers including OpenAI, Anthropic, Groq, Cohere, Ollama, DeepSeek, Cloudflare, Google, Mistral, MiniMax, Moonshot, Nvidia, and llama.cpp.docs.inference-gateway.com · read 10 Oct 2026
- Provider switching
- The gateway lets applications switch providers by changing configuration, without an application redeploy, and can route model aliases to pools of upstream deployments.docs.inference-gateway.com · read 9 Oct 2026
- Purpose
- Inference Gateway is an open-source, cloud-native proxy that provides one OpenAI-compatible API for multiple hosted and local LLM providers.docs.inference-gateway.com · read 10 Oct 2026
- Routing
- The gateway supports model routing so a stable model alias can point to a pool of upstream deployments.docs.inference-gateway.com · read 10 Oct 2026
- SDKs
- Official SDKs are available for Python, TypeScript, Go, and Rust, with streaming support over Server-Sent Events.docs.inference-gateway.com · read 10 Oct 2026
