agentgateway

agentgateway is a self-hosted HTTP and gRPC gateway for controlling traffic among services, language models, MCP servers, and agent runtimes. Its LLM gateway presents an OpenAI-compatible API for providers including OpenAI, Anthropic, Bedrock, Gemini, Vertex, and OSS Llama runs. For self-hosted inference, it routes workloads on vLLM, TGI, and Triton using latency, cost, and model considerations. MCP capabilities include server discovery, scoped tool calls, and audit trails; A2A routing connects runtimes such as LangChain, CrewAI, and ADK. Security options include JWT and OIDC, API keys, authorization policies, TLS and mTLS with certificate rotation, and rate limits. Guardrails can inspect MCP calls and responses for prompt injection or unsafe payloads, while prompt guards provide checks and redaction. Observability features include OpenTelemetry, Prometheus token-usage histograms, Jaeger traces, and request logs with latency, token counts, and realized USD cost. Teams can set token and dollar spending caps by key or team and review a dashboard grouped by model, provider, and user. It is free and deploys as a binary, Docker container, or on Kubernetes.

agentgateway brings routing, security, observability, and spending controls into one self-hosted gateway. Its free plan may suit teams that want those controls for service, LLM, MCP, and agent traffic.

It suits platform teams and tool-builders managing model, MCP, or agent traffic in self-hosted environments. AI enthusiasts are also invited to join the project community.

agentgateway is a self-hosted control point for teams running services, language models, MCP tools, and agent-to-agent workflows. It suits platform engineers and tool-builders who need one policy and telemetry layer across mixed AI infrastructure. The trade-off is operational responsibility: this is infrastructure software, not a managed endpoint.

Overview

Built as an open-source HTTP and gRPC gateway, agentgateway combines traffic routing with security, observability, and spend governance. Its scope reaches from conventional service traffic to model inference and agent protocols, making it a strong fit for organizations standardizing AI access on infrastructure they operate. Teams that want a vendor-hosted gateway or a turnkey user interface should consider another category of product.

Key features

Model and agent routing

An OpenAI-compatible API fronts providers such as OpenAI, Anthropic, Bedrock, Gemini, Vertex, and OSS Llama deployments. For self-hosted inference, routing can account for latency, cost, and model characteristics across vLLM, TGI, and Triton. That breadth is useful when workloads span cloud APIs and privately run models, while the compatibility layer can reduce provider-specific integration work.

MCP server discovery, scoped tool calls, and audit trails address the operational side of tool use. A2A routing connects runtimes including LangChain, CrewAI, and ADK, so teams can place policy between cooperating agents rather than treating each runtime as an isolated system.

Security and guardrails

JWT and OIDC, API keys, authorization policies, TLS and mTLS with certificate rotation, and rate limiting provide a substantial access-control baseline. MCP guardrails inspect calls and responses for prompt injection and unsafe payloads; prompt guards add checks and redaction. These mechanisms support a defensible policy layer, but they still require operators to design rules and maintain certificates and identities.

Telemetry and spending

OpenTelemetry is enabled by default, with Prometheus token-usage histograms, Jaeger traces, and request logs that include latency, token counts, and realized USD cost. Scoped virtual keys and per-key or per-team token and dollar caps turn usage data into enforceable budgets, while dashboards group spend by model, provider, and user. This is especially valuable for platform teams accountable for unpredictable model consumption.

Deployment and integrations

The gateway is offered as a standalone binary, Docker image, or Kubernetes deployment. Listed model integrations include Azure OpenAI, Mistral, DeepSeek, Ollama, xAI, and Snowflake in addition to the providers above. Canary routing, TLS automation, rate limiting, and authorization policies are included capabilities, and ingress supports both API models.

Pricing

agentgateway (open source): free is the free, open-source gateway. It can run as a binary, in Docker, or on Kubernetes, with self-hosted deployment and no seat or usage quota stated. That makes it suitable for experimentation and for teams prepared to fund their own operations. The zero price does not provide a managed service; commercial support is offered through partners, including a Solo.io option with 24×7 coverage, upstream collaboration, and best-practice guidance.

Platforms

agentgateway targets API and Linux environments and is self-hosted. Deployment choices cover a standalone binary, Docker, and Kubernetes, so it can fit both a single infrastructure host and a cluster-based platform. Its ingress configuration supports both API models, with TLS automation and canary routing available.

Who it's for

Choose it if you are a platform engineer, tool-builder, or AI-focused infrastructure team consolidating provider access, agent protocols, security policy, telemetry, and budgets. It is less appropriate for a small application team that wants an immediately managed gateway, or for organizations unwilling to operate networking, identity, certificates, and policy themselves.

Pros and cons

• Pro: One OpenAI-compatible front door spans major hosted providers and self-hosted inference targets, easing mixed-provider architecture.
• Pro: MCP and A2A controls include scoped calls, discovery, audit trails, and runtime routing rather than stopping at ordinary HTTP ingress.
• Pro: Token-aware telemetry and per-key or per-team dollar caps make model spending measurable and enforceable.
• Con: Self-hosting shifts deployment, upgrades, identity, certificates, and policy maintenance to your team.
• Con: The free plan has no stated managed-service entitlement, seat allowance, or usage quota, so operating cost and capacity planning remain your responsibility.

Alternatives

For a broader survey of ingress products, see Kubernetes Ingress Controllers.

• Airlock Microgateway is a freemium, self-hosted API option with a free Community Edition for small environments or local development, but throughput is enforced and its license is valid for six months.
• NGINX Ingress Controller is a free self-hosted choice when you specifically need a Kubernetes cluster and the NGINX Open Source edition.
• ngrok fits readers wanting API, desktop, or web access and a free allowance of up to three online endpoints and 1GB of transfer.
• BFE Ingress Controller is a free Apache 2.0-licensed alternative with no stated plan limits.
• Contour is a free open-source Kubernetes ingress controller with community support.
• Envoy Gateway is a free Apache-2.0 open-source project for teams centered on Envoy-based gateway infrastructure.
• F5 BIG-IP Container Ingress Services is free software when you already have a compatible licensed BIG-IP system, plus Kubernetes or OpenShift and AS3.
• HAProxy Ingress is a free Linux, self-hosted option that requires Kubernetes; version 0.16 requires Kubernetes 1.21 or newer.

Verdict

agentgateway is the right choice for platform teams that need unified routing, AI-specific guardrails, deep telemetry, and enforceable model budgets on infrastructure they control. Its main reason to choose it is the combination of LLM, MCP, and A2A governance in one free gateway. Look elsewhere if you need a managed service or do not want to carry the operational work of self-hosting.