Self-host agentgateway
v1.7.0AI-native gateway for LLM, MCP and agent traffic
$ docker run -d --name agentgateway --user "$(id -u):$(id -g)" -v "$PWD/agentgateway-config:/config" -p 4000:4000 cr.agentgateway.dev/agentgateway:v1.6.0- Stars
- 6 180
- Ports
- 4000 (Default gateway port; the UI is served on it when attached to that gateway), 15000 (Admin interface, loopback-only by default)
cr.agentgateway.dev/agentgatewayWhat is agentgateway?
agentgateway is an Apache-2.0 licensed gateway written in Rust that speaks AI-native protocols such as MCP and A2A alongside ordinary HTTP, gRPC and TCP traffic, giving platform teams one data plane for agent-to-LLM, agent-to-tool and agent-to-agent calls. Routing covers major LLM providers behind a single OpenAI-compatible API with load balancing and failover, MCP servers federated into one endpoint over stdio, HTTP, SSE or Streamable HTTP, A2A agent meshes, and latency/cost-aware inference routing to self-hosted model replicas. Governance features include JWT and OIDC authentication, API keys and basic auth, external authorization, CEL-based RBAC, CORS/CSRF, TLS and mTLS, local and remote rate limiting, prompt and tool-call guardrails (regex, OpenAI moderation, Bedrock Guardrails, Google Model Armor, webhooks) and OpenTelemetry metrics, traces and logs with token and cost accounting. Operators install it with a curl script that drops the binary, run the official container image (default gateway on port 4000, admin interface on localhost:15000), deploy a Helm chart on Kubernetes, or use the Kubernetes control plane with Gateway API resources in standalone or Kubernetes mode. Configuration is a flat YAML file that is watched and reloaded live, or xDS/Kubernetes CRs; a built-in admin UI with an LLM playground can edit config when an SQLite or PostgreSQL database is enabled. An enterprise distribution and support are sold separately by Solo.io; the project is now governed by the Agentic AI Foundation under the Linux Foundation.
Should you self-host agentgateway?
agentgateway scores 76/100 on selfhost.directory (how scoring works) and ranks #3 of 38 in API Gateway. It has 6 180 GitHub stars, +903 in the last 30 days.
It shipped 9 stable releases in the past year. The latest, v1.7.0, came out on 27 October 2026. New versions land roughly every 4 weeks.
Setup is easy: most people get it running in one sitting. An official Docker image is available. It is Apache-2.0-licensed and written mostly in Rust.
Its closest alternative on selfhost.directory is OmniRoute (88/100, 87 254 stars). It scores higher overall, so compare the two before you pick. See agentgateway vs OmniRoute side by side.
Features
- A2A agent-to-agent gateway
- Authentication and authorization
- Built-in admin UI
- Cost and budget controls
- Dynamic configuration updates
- Guardrails / content filtering
- Inference Routing
- LLM Gateway
- LLM Playground
- MCP gateway with tool federation
- Observability
- Rate Limiting
- Service gateway (HTTP, gRPC, TCP)
- TLS and mTLS
agentgateway README
How to install agentgateway
Install agentgateway with Docker
mkdir agentgateway-config
docker run -d \
--name agentgateway \
--user "$(id -u):$(id -g)" \
-v "$PWD/agentgateway-config:/config" \
-p 4000:4000 \
cr.agentgateway.dev/agentgateway:v1.6.0
Generates a default config.yaml and SQLite database in the mounted /config directory; UI at http://localhost:4000/ui.
Install agentgateway with Docker Compose
services:
agentgateway:
container_name: agentgateway
restart: unless-stopped
image: cr.agentgateway.dev/agentgateway:v1.6.0
# Replace with your user and group IDs, such as the output of: id -u && id -g
user: "1000:1000"
ports:
- "4000:4000"
volumes:
- ./agentgateway-config:/config
mkdir agentgateway-config
docker compose up -d
Same approach as docker run; the user value must be your own user and group IDs so the container can write to the mounted directory.
Install agentgateway with Kubernetes
helm upgrade -i --create-namespace \
--namespace agentgateway-system \
--version v1.6.0 agentgateway-crds oci://cr.agentgateway.dev/charts/agentgateway-crds
Kubernetes mode installs two charts, CRDs plus control plane, and is configured with agentgateway custom resources and Gateway API.
Install agentgateway with Linux
curl -sL https://agentgateway.dev/install | bash
Installs the agentgateway binary into /usr/local/bin; specific versions via --version, nightly builds from GitHub Actions artifacts.
Install agentgateway with Helm
helm upgrade -i agentgateway-standalone \
oci://cr.agentgateway.dev/charts/agentgateway-standalone \
--namespace agentgateway-system \
--version v1.6.0 \
-f values.yaml
Standalone chart for a single Kubernetes Deployment; the chart installs no database, so point it at your own PostgreSQL or run readonly.
How to configure agentgateway
config.database.urlDatabase connection URL for the analytics, logs and UI storage; a postgres:// or postgresql:// URL selects PostgreSQL, anything else is a SQLite file path.values: sqlite://... or postgres://...
replicaCountpodDisruptionBudget.enabledmonitoring.enabledextraEnv
agentgateway releases & version history
AI summary Enhanced model insights and routing capabilities
🎉 Welcome to the 1.7.0 release of the agentgateway project!
Agentgateway 1.7 knows more about your models and your traffic than ever. The model catalog now knows each model's context window, modalities, and capabilities, and shares them through /v1/models and CEL. Costs reflect the service tier that actually served each request. New analytics break down tokens, cost, cache hit rate, and latency by model, user, or anything else you log.
On the routing side, agentgateway speaks OpenAI Decisions and TypeSafe SystemOne natively, adds Perplexity, brings Copilot to Kubernetes, and lets virtual models choose a target by calling your own routing service. Guardrails now cover tool output from every vendor, and MCP traffic can be filtered with plain CEL.
Rounding it out are automatic agent session tracking, OIDC that works for both browsers and API clients, a new agctl standalone CLI, and a UI that lets you edit anything as raw YAML.
Before upgrading, review the breaking and behavior changes below. In particular, standalone deployments now always encrypt MCP sessions, prompt guards inspect tool outputs by default, and dynamic backends can no longer reach loopback or link-local addresses.
Artifacts
Docker images are available:
cr.agentgateway.dev/agentgateway:v1.7.0cr.agentgateway.dev/controller:v1.7.0
Helm charts are available:
cr.agentgateway.dev/charts/agentgateway:v1.7.0cr.agentgateway.dev/charts/agentgateway-crds:v1.7.0cr.agentgateway.dev/charts/agentgateway-standalone:v1.7.0
Binaries are available below.
Quick Start
Follow the Kubernetes or Standalone quick start guide to get started.
🔥 Breaking changes
Standalone always encrypts MCP sessions
Agentgateway allows defining an MCP session key that encrypts MCP session IDs. On Kubernetes, this was auto-generated; standalone deployments now match this.
When config.session.key is unset, agentgateway generates a key and stores it in the config file or database, whichever is writable. If neither is writable, agentgateway logs a warning and asks you to set the key.
If you previously had session encryption enabled, MCP sessions established before the upgrade are rejected, and clients must establish new sessions. To keep sessions unencrypted, set config.session.key: disabled.
Prompt guards inspect tool outputs by default
All prompt guards except webhooks now honor scope, including OpenAI Moderation, Google Model Armor, and Azure Content Safety. The default request scope now also includes ToolOutput, so tool results are inspected by every guard that uses the default scope.
This can produce new rejections or masking, and increases the content sent to vendor guardrail APIs. To retain the 1.6 behavior, set the scope explicitly:
scope: - SystemPrompt - Messages
Webhook guards do not support scope and always inspect the system prompt and messages.
CEL header lookups return every value of a repeated header
request.headers["x"] and similar lookups now return a list when a header appears more than once. Previously this depended on whether the expression could be optimized (internally), so the same lookup could return a single value in one expression and a list in another depending on the exact syntax. Expressions that read headers which only appear once are unaffected.
Dynamic backends cannot connect to loopback or link-local addresses
Dynamic backends now reject destinations that resolve to loopback, link-local (such as cloud metadata servers), or unspecified addresses. Set config.backend.allowDynamicLoopback: true if a dynamic backend must reach a service local to the gateway.
Error responses no longer include internal details
Error responses sent to clients no longer include internal details such as addresses, file paths, CEL expressions, or upstream error text. The full error is still logged, and status codes are unchanged.
Some examples:
| Before | After |
|---|---|
upstream tcp call failed: tcp connect to 10.12.4.7:5432 failed: Connection refused |
connection refused |
authentication failure: no bearer token found |
authentication failed |
policy failed: cel expression ... failed: no such key |
policy failed |
failed to process LLM response: failed to parse response: ... |
failed to process LLM response |
Removed
- Messages-to-Completions preference:
AGENTGATEWAY_MESSAGES_PREFER_COMPLETIONShas been removed, as planned in 1.6. Anthropic Messages requests always prefer the OpenAI Responses format when a provider supports both. For OpenAI-compatible servers that do not implement/v1/responses, configure a custom provider that advertises only Chat Completions.
🔄 Other behavior changes
-
Normalized service tier:
llm.serviceTieris now one ofstandard,flex,priority, orreserved, regardless of the provider's own naming. Update CEL expressions and dashboards that compare against provider-specific values such asdefault. -
Request log database migration: On startup, standalone request log databases (SQLite or PostgreSQL) gain new analytics columns that are backfilled from existing rows. This can take some time on large databases.
-
Bedrock
tool_choice:tool_choicenow reaches Bedrock Converse; it was previously dropped silently. Requests that force tool use on models that do not support it, such as with extended thinking, now return an error from Bedrock. -
Token rate limits for non-chat APIs: Token-based LLM rate limits now count usage from
Embeddings,Rerank,Decisions, andSystemOnerequests. -
LLM buffer size: The 32 MiB default LLM request buffer now applies before request policies run instead of after.
-
Webhook guardrail responses: A webhook
passresponse that contains any field other thanreasonis now an error instead of passing unmasked content. -
Google Model Armor errors: A non-success response from Model Armor is now a guard failure and follows the guard's
failureMode. -
Backend mirrors: Request mirroring runs after backend policies, so requests rejected by backend authorization are no longer mirrored.
-
AgentgatewayModelwildcard precedence: On Kubernetes, an exact model name such asopenai/gpt-5.5now wins over a wildcard such asopenai/*, matching standalone. -
MCP OpenAPI paths: A non-root path in an OpenAPI MCP target's host URL or
pathfield now overrides the spec's server prefix. Previously the path was ignored. -
MCP OpenAPI responses: Non-JSON responses are returned as typed MCP content (text, image, audio, or blob) instead of a
{code, message}wrapper. -
MCP session errors: Session IDs that cannot be decoded return
404instead of400, so clients start a new session after a key change. -
gRPC health: Errors that agentgateway itself returns to gRPC clients now record their
grpc-status, and only server-side gRPC errors mark a backend unhealthy, matching the treatment of5xxand4xxfor HTTP. -
gRPC policy calls: A gRPC policy call, such as external processing, that fails before reaching the service now reports
UNAVAILABLEinstead ofUNKNOWN. -
Invalid OAuth configuration: Invalid token exchange or Cross App Access configuration no longer rejects the whole xDS update. The affected policy fails closed and reports
Accepted=False. -
Anthropic version header: The client's
anthropic-versionheader is forwarded instead of being replaced.
🌟 New features
LLM APIs and providers
- OpenAI Decisions and TypeSafe SystemOne: Agentgateway supports the OpenAI Decisions API and the TypeSafe SystemOne API natively, translates between them in both directions, and includes a native TypeSafe provider. Both work on standalone and Kubernetes, and on Azure Foundry.
- Audio APIs:
/v1/audio/transcriptions,/v1/audio/translations, and/v1/audio/speechare recognized LLM endpoints. Duration-billed models such aswhisper-1reportllm.audioSecondsand can be priced per minute with the newperAudioMinutecatalog rate. - Perplexity provider: Perplexity is available as a built-in provider.
- Copilot on Kubernetes:
AgentgatewayBackendandAgentgatewayModelcan use the Copilot provider with credentials from a Kubernetes Secret. - Native Bedrock Runtime APIs (preview): Set
AGENTGATEWAY_BEDROCK_RUNTIME_CONVERSE=falseto send chat requests to Bedrock Runtime's native Anthropic Messages and OpenAI Chat Completions and Responses endpoints, like Mantle, instead of translating them to Converse. This avoids format conversion for clients such as Claude Code and Codex. Converse remains the default for now. - Callout router for virtual models: A virtual model can select its target model by calling an external HTTP service, on both standalone and Kubernetes.
- Per-model HTTP transformations: Provider defaults can set
requestTransformationto modify headers and metadata of requests to a specific model. - More accurate token counting: A new tokenizer estimates token counts more accurately, runs about 10x faster, and uses less memory.
- Better format conversion: Reasoning from vLLM and OpenRouter is preserved when converting Chat Completions to Messages, Bedrock reasoning streams as reasoning items in Responses, Bedrock redacted reasoning is preserved, and Vertex Gemini accepts
https://URIs in file content.
Model catalog and cost
The model catalog now describes models as well as pricing them. Entries include display name, release date, lifecycle status, token limits, modalities, capabilities, and reasoning controls.
/v1/models returns this metadata, so clients can discover what each model supports:
{
"id": "gpt-5.5",
"object": "model",
"owned_by": "openai",
"name": "GPT-5.5",
"context_window": 1050000,
"max_input_tokens": 922000,
"max_tokens": 128000,
"architecture": {"input_modalities": ["text", "image", "file"], "output_modalities": ["text"]},
"capabilities": {"reasoning": true, "structured_outputs": true, "tools": true, "temperature": false},
"supported_parameters": ["max_tokens", "reasoning", "reasoning_effort", "response_format", "tool_choice", "tools"]
}The same metadata is available in CEL as llm.model, such as llm.model.limits.context.
Pricing now accounts for the service tier that served each request. Agentgateway records the tier for OpenAI, Anthropic, Bedrock, Vertex, and Gemini as llm.serviceTier, and catalog pricing tiers can set serviceTier alongside contextOver, so flex or priority requests are costed at their actual rates. The built-in catalog includes Vertex service-tier prices; rates for other providers can be configured manually.
tiers: - serviceTier: priority rates: {input: "2.5", output: "15"} - serviceTier: priority contextOver: 272000 rates: {input: "5", output: "22.5"}
- 1-hour cache pricing: Anthropic cache writes with a 1-hour TTL are priced separately using the new
cacheWrite1hrate. - Budgets without a window: An API-key budget without a
windowaccumulates usage for its lifetime, providing a fixed spending limit. - More CEL during request policies:
llmRequestandllm.requestModelare available to route-level request policies, before LLM processing.
Guardrails
-
Scope for vendor guardrails: OpenAI Moderation, Google Model Armor, and Azure Content Safety support
scope, like regex and Bedrock Guardrails. -
Reasoning and response scope: Guards scan reasoning and thinking content, and response prompt guards support
scope. -
Rejection headers on Kubernetes: Kubernetes prompt guard rejection responses can add, set, and remove headers.
-
CEL processor for MCP guardrails:
mcpGuardrailscan reject or transform MCP requests and responses with CEL, without an external service.mcpGuardrails: processors: - kind: cel methods: {"tools/call": request} condition: mcp.params.name == 'restricted_tool' reject: "This tool is unavailable"
Agent sessions
Agentgateway detects the agent session that a request belongs to from well-known agent headers, such as x-claude-code-session-id, and records it in logs and in CEL as request.agent.session. Set config.standardAttributes.session to a CEL expression to identify sessions yourself.
Security and identity
- OIDC with bearer tokens: OIDC policies can set
credentials: sessionOrBearerto accept a bearer ID token in addition to a browser session, so one policy covers both browser and API clients.credentials: permissivekeeps browser login while letting other requests fall through to other authentication policies. - Backend credentials from CEL: Backend authentication keys can be computed from a CEL expression, such as a value returned by an external authorization server.
- MCP authentication with token passthrough: MCP authentication works with JWT
preserveToken: true. - Callouts through a proxy: Callouts such as JWKS fetches, OIDC discovery, external authorization, and cloud credential fetches can be tunneled through an HTTP proxy with
config.callouts.tunnelor the standardHTTP_PROXYenvironment variables.
MCP
- Conditional targets on Kubernetes: MCP conditional targets are available in the Kubernetes CRDs.
- Session cleanup across replicas:
DELETErequests resume the session on any replica so that upstream sessions are closed. - Idle timeouts: MCP requests respect the route's idle timeout.
Kubernetes and operations
- Shared HTTPS and TLS ports: HTTPS and TLS passthrough listeners can share a port on the same Gateway, selected by SNI.
- Optional RBAC: Set
rbac.create=falseto install the Helm chart without RBAC resources. - Native histogram scraping: The PodMonitor and ServiceMonitor in both Helm charts support native histogram scraping, relabelings, and metric relabelings.
- Graceful HTTP/1 drain: HTTP/1 connections drain more gracefully during shutdown.
- Faster controller: The controller does less work when unrelated resources change, and status and xDS output are more stable, reducing unnecessary writes and pushes.
CLI and UI
-
agctl standalone: Manage a standalone agentgateway through its admin API withgetandapply, including API keys, models, policies, and MCP targets. -
Richer analytics: The request log database records cache read and write tokens, reasoning tokens, time to first token, and time per output token.
agctl standalone analyticssummarizes requests, errors, tokens, cost, time to first token, tokens per second, and cache hit rate, grouped by provider, model, status, user, or any logged attribute, and can bucket results by time or into latency histograms:agctl standalone analytics --by provider,model --since 168h agctl standalone analytics --by agentgateway.user --provider openai agctl standalone analytics --by model,ttft -
agctl bug-report: Collect diagnostic information into a single archive for sharing. -
Raw editing: Every form editor has a raw YAML mode, and fields the form does not support are preserved on save.
-
Clearer forms: Required fields and schema help are shown consistently, with errors reported in place.
-
Settings page: A top-level settings page groups gateway-wide settings.
-
Client Setup in xDS mode: Client Setup is available for models served in xDS mode, and includes a recipe for Continue.
-
More UI improvements: API keys can be rotated with an auto-generated value, and the Users and Groups log filters are searchable.
🪲 Notable fixes
- Fixed custom providers with Detect or Passthrough formats duplicating the base path, such as
/v1/v1/.... - Fixed Gemini tool schemas that used
oneOfreturning empty tool calls, and counted Gemini reasoning tokens consistently with other providers. - Fixed Bedrock dropping cache markers on unsupported content and mid-stream system messages invalidating the prompt cache.
- Fixed passthrough failing on SSE streams with an unterminated final event.
- Fixed Azure prompt-cache token conventions.
- Fixed MCP conditional targets being evaluated only when a session was initialized instead of on each request, and resumed sessions keeping outdated settings after a configuration change.
- Fixed MCP rejecting
Acceptvalues sent as separate header lines. - Fixed MCP using stale information after a transformation, and a double
//in some issuer URLs. - Fixed requests answered locally, such as unknown models or model authorization failures, missing request data in access logs and CEL.
- Fixed
AgentgatewayModelreferences from another namespace ignoring ReferenceGrants. - Fixed policies on unmanaged Gateways being reported as attached.
- Fixed a route budget replacing a Gateway budget.
- Fixed the standalone external authorization default body.
- Fixed numeric keys in configuration serialization.
- Fixed the controller resetting the Istio log level.
- Fixed transparent huge pages not being disabled, which increased memory usage.
- Fixed the UI MCP and LLM playgrounds implicitly sending the logged-in session's credentials.
Contributors
Thank you to everyone who contributed code, reviews, documentation, bug reports, and CI improvements for this release!
What's Changed
- llm: drop unsupported reasoning on old openai models by @howardjohn in #3744
- fix(controller): report policies on unmanaged Gateways as not attached by @AhmadMasry in #3572
- llm: do not drop unknown cache control blocks by @howardjohn in #3759
- bedrock: handle mid-stream system messages by @howardjohn in #3764
- examples: do not pin versions to avoid cve spam by @howardjohn in #3765
- docs: add protocol requirement onto MCP field by @howardjohn in #3768
- Bump node deps by @howardjohn in #3767
- llm: add perplexity provider by @howardjohn in #3771
- Allow Helm installations without creating RBAC resources by @shubhamojha1 in #3763
- gemini: properly count reasoning tokens by @howardjohn in #3773
- ui: add confirmation when changing where we serve it by @howardjohn in #3772
- mcp: consistently avoid double // in issuer url by @howardjohn in #3774
- Disable GHA go caching by @howardjohn in #3779
- ci: force cached images by @howardjohn in #3780
- go mod stickydisk by @howardjohn in #3782
- Update model catalog by @github-actions[bot] in #3787
- Unexpected HTTP 406 when passing separate
Acceptheader lines instead of comma-separated values to the MCP endpoint by @shubhamojha1 in #3786 - Do not rebuild tools by @howardjohn in #3781
- schema gen: link to reference by @howardjohn in #3800
- ci lint: use optimized cache for proxy-lint job by @howardjohn in #3801
- vendor Accept header negotiation by @howardjohn in #3798
- ui: add top level settings page with sub-pages by @howardjohn in #3803
- test(mcp): pin upstream JSON-RPC error code propagation by @Yoel-qin in #3762
- build(deps): bump the cargo-weekly group with 11 updates by @dependabot[bot] in #3795
- deployer: create session key Secrets once and never overwrite them by @markuskobler in #3802
- llm: expose virtual model routing decisions in logs and traces by @rajatvig in #3695
- osv scanner rmcp cleanup by @jenshu in #3819
- Update model catalog by @github-actions[bot] in #3814
- feat(helm): support native histogram & relabelings by @maximumG in #3797
- llm: make Bedrock Converse honor tool_choice by @choyongho-bart0401 in #3812
- build: check out ui files with LF line endings by @TheSeydiCharyyev in #3808
- ui: show Client Setup in XDS mode by @TheSeydiCharyyev in #3634
- crypto: enforce FIPS policy for JWT signing and verification by @ymesika in #3813
- guardrail matches consecutive phone numbers by @stevenctl in #3737
- mcp: port conditional targets to kubernetes CRD by @stevenctl in #3824
- examples: add integration example with headroom by @howardjohn in #3822
- llm: allow per-model http transformations by @howardjohn in #3823
- improve bedrock cache preservation for dropped content blocks and anthropic ttl by @stevenctl in #3721
- auth: allow extracting a key from a CEL expression by @howardjohn in #3821
- ui: expose proper gateway url for client setup by @howardjohn in #3807
- fix(mcp): return non-JSON OpenAPI responses as typed MCP content by @lewis-treacy in #3815
- (breaking) cel: consistently expose headers optimized vs unoptimized by @howardjohn in #3778
- metrics: better negotiation of content type by @howardjohn in #3746
- fix(bedrock): preserve streamed redacted reasoning by @dev-ahuja-gl in #3776
- mcpGuardrails: CEL processor by @stevenctl in #3711
- scan reasoning/thinking and add scope to response promptGuards by @stevenctl in #3732
- mcpGuardrails: port expression to k8s by @stevenctl in #3830
- llm: consistently derive operation name by @howardjohn in #3831
- llm: add first class OpenAI decisions support by @howardjohn in #3832
- virtual model: add callout router by @howardjohn in #3769
- fix: report the running config from /api/config/effective by @Yoel-qin in #3692
- Update model catalog by @github-actions[bot] in #3837
- ci: bump kind node image to Kubernetes v1.37.0 by @markuskobler in #3846
- refactor: cleanup dead code by @howardjohn in #3847
- build(deps): bump the cargo group across 1 directory with 2 updates by @dependabot[bot] in #3849
- Fail closed on invalid OAuth xDS configuration by @markuskobler in #3809
- feat(k8s): support headers on prompt guard rejection responses by @uc4w6c in #3810
- refactor: Allow policies to be split out of core crate by @howardjohn in #2764
- standalone helm: enable session key configuration by @howardjohn in #3842
- Automatically detect and report 'session' by @howardjohn in #3850
- proxy: gracefully handly HTTP/1 draining by @howardjohn in #3833
- Support translating from OpenAI Decisions to TypeSafe SystemOne format by @howardjohn in #3839
- Add a native typesafe preset provider by @howardjohn in #3852
- Update model catalog by @github-actions[bot] in #3856
- llm: fix custom providers with Detect/Passthrough by @howardjohn in #3853
- Add Kubernetes support for typesafe and decisions APIs by @howardjohn in #3854
- cel: cleanup resolver implementations by @howardjohn in #3848
- misc CI improvements by @howardjohn in #3862
- fix(bedrock): stream reasoning as reasoning items in responses by @howardjohn in #3828
- virtual model: kubernetes callout support by @howardjohn in #3843
- fix(controller): do not reset istio log level in NewControllerBuilder by @noony in #3817
- ext_proc: fail a local error of a gRPC policy call with its gRPC status by @QuentinBisson in #3793
- resource_manager: files read while parsing become managed dependencies (fork) by @howardjohn in #3863
- Fix/substrate resume trace propagation by @linsun in #3674
- add AgentgatewayModel GK to default known from references by @conradhanson in #3869
- Add source repository to Chart.yaml by @lareeth in #3871
- Allow sending callout requests through CONNECT tunnel by @howardjohn in #3864
- Allow substrate egress while actors resume by @EItanya in #3882
- Update model catalog by @github-actions[bot] in #3877
- xds: sort policies for stability by @howardjohn in #3885
- controller: keep status entry order stable across controllers by @rajatvig in #3883
- oidc: allow direct bearer tokens by @howardjohn in #3826
- mcp: avoid using a stale cached MCPInfo by @howardjohn in #3893
- ui: avoid implicit passing creds to MCP and LLM playground by @howardjohn in #3892
- guard: make phone number parser a bit stricter by @howardjohn in #3890
- oidc: explicitly tell clients to avoid caching refresh token flow by @howardjohn in #3895
- llm: better int type casting by @howardjohn in #3887
- Add dockerfile for oss-scanner by @howardjohn in #3904
- perf: re-disable THP by @howardjohn in #3902
- Update model catalog by @github-actions[bot] in #3905
- jwks: support x5c key sets in fips140=only mode by @ymesika in #3924
- llm: rewrite oneOf to anyOf in Gemini schema normalization by @rajatvig in #3920
- budget: do not override gateway policy from a route policy by @howardjohn in #3888
- controller: align status builders with the syncer and fix ancestor matching by @rajatvig in #3911
- fix(ui): remove useKeyWithClickEvents suppressions by @iandvt in #3906
- fix(ui): use headings for read-only YAML sections by @iandvt in #3910
- fix(ui): use a button for log turn help by @iandvt in #3913
- fix(ui): use buttons for Field help controls by @iandvt in #3914
- llm: fail close on model armor error by @howardjohn in #3925
- feat(ui): search the values offered by the user and group filters by @erentorlak in #3919
- llm: support service tier pricing by @howardjohn in #3661
- xbackend: avoid collisions with AGWBackend keys by @howardjohn in #3891
- build: always include UI, just make it optional by @howardjohn in #3929
- Fix numeric key serialization in config by @howardjohn in #3933
- respect idle timeout on MCP path by @howardjohn in #3894
- allow mcp auth with jwt validation by @stevenctl in #3936
- health: only consider server-side grpc errors as unhealthy by @howardjohn in #3897
- llm: fix cache convention on azure by @howardjohn in #3899
- llm: respect buffer limits on various paths by @howardjohn in #3896
- refactor(http): extract shared strip_path_prefix for segment-aware prefix matching by @dongjiang1989 in #3874
- fix: Redact database URL when serializing the config. by @chrisgavin in #3878
- Bump go dependencies by @howardjohn in #3865
- Better token counting by @howardjohn in #3867
- feat: allow to auto-gen a key on replacing by @guenhter in #3923
- ui: fallback if we cannot gracefully edit the yaml by @howardjohn in #3866
- examples: add OpenShell integration by @danehans in #3908
- ui: fix CEL playground sample context serviceTier by @ymesika in #3950
- helm: fix Prometheus relabeling doc links in values comments by @kristin-kronstain-brown in #3951
- fix(llm): don't fail passthrough on an unterminated final SSE event by @johnsto in #3676
- examples: simplify single-runtime tier routing by @danehans in #3944
- fix(ui): drop provider icon noDescendingSpecificity suppressions by @iandvt in #3943
- xds: report proxy configuration warnings separately from rejections by @markuskobler in #3953
- build(deps): bump the cargo-weekly group with 9 updates by @dependabot[bot] in #3946
- fix(proxy): record grpc-status for trailers-only error responses by @QuentinBisson in #3873
- Update model catalog by @github-actions[bot] in #3945
- Bump istio and adopt performance improvements by @howardjohn in #3954
- ext authz: fix default body on standalone by @howardjohn in #3940
- Update model catalog by @github-actions[bot] in #3956
- ui: preselect first model for new virtual model targets by @howardjohn in #3939
- AgentgatewayModel: proper wildcard precedence by @howardjohn in #3932
- perf: cache proxy TLS configs by @howardjohn in #3926
- respect client anthropic-version header by @stevenctl in #3957
- examples: simplify Datadog integration by @danehans in #3935
- oidc permissive mode by @jenshu in #3903
- fix(ui): associate Field labels with their controls by @iandvt in #3915
- llm/vertex-gemini: support https:// URIs in file content parts by @matvas-txf in #3691
- fix(ui): reorder styles to drop 29 noDescendingSpecificity suppressions by @iandvt in #3961
- catalog: Import more data into the catalog by @howardjohn in #3960
- fix(ui): reorder styles to drop 13 noDescendingSpecificity suppressions by @iandvt in #3969
- Fix Webhook guardrail: MaskAction with extra message fields silently pa… by @lcopi in #3964
- llm: document agentgateway extensions to standard APIs by @howardjohn in #3959
- examples: fix token exchange docs to match config schema by @markuskobler in #3967
- cel: fix non-this indexing by @howardjohn in #3966
- feat(copilot): add kubernetes provider support by @iandvt in #3605
- llm: price 1h cache writes separately from 5m by @liorbenita2 in #3825
- catalog: render model information into /v1/models by @howardjohn in #3965
- Fix bad rebase in model catalog by @howardjohn in #3982
- mcp: resume sessions on DELETE so upstream sessions get closed by @markuskobler in #3976
- deps: avoid depending on GIE conformance in go.mod by @howardjohn in #3978
- ui: Fixes around policy editors by @howardjohn in #3979
- fix(copilot): restore protocol headers for generic backends by @iandvt in #3983
- test: fix flakey extauthz test by @howardjohn in #3984
- cel: expose model catalog info by @howardjohn in #3981
- catalog: add prices for anthropic 1h cache by @howardjohn in #3977
- Close drift on recent substrate contract changes by @keithmattix in #3985
- Add ui-maintainers as ui code owner by @Charlesthebird in #3958
- Fix Substrate worker identities and default egress policy lookup by @EItanya in #3990
- fix(ui): complete hook dependencies outside Logs.tsx by @iandvt in #3988
- config store: reject writes validated against a stale snapshot by @jbohanon in #3656
- controller: Drop metrics that are expensive and not useful. by @howardjohn in #3992
- ui: minor fixes for monaco by @howardjohn in #3993
- ui: add 'raw' mode for all form editors by @howardjohn in #3994
- fix(ui): complete Logs hook dependencies by @iandvt in #3997
- feat(standalone): honor mcp openapi path config by @jbohanon in #3973
- budget: allow budgets with no refresh by @howardjohn in #3970
- hbone: disable data frame budget by @howardjohn in #3952
- ui: elide defaults from saved config by @howardjohn in #3998
- llm: increase buffer size before request policies by @howardjohn in #4002
- ui: proper and consistent required field markers by @howardjohn in #4004
- ui: include backend policies in MCP server page by @howardjohn in #4005
- ui: save every editor through useMergedEdits by @howardjohn in #4008
- ui: mark required fields on the remaining forms; raw mode for keys by @howardjohn in #4013
- proxy: move backend mirrors to after backend policies by @howardjohn in #4011
- ui: report all OIDC required fields on the first save by @howardjohn in #4014
- llm: add systemone -> decision translation by @howardjohn in #3928
- Update model catalog by @github-actions[bot] in #4023
- fix(ui): remove static interaction ignores from startup and tooltip by @iandvt in #4022
- fix(ui): remove static interaction ignores from shared controls by @iandvt in #4021
- dep bumps by @markuskobler in #4028
- feat(azure): support Foundry Decisions and SystemOne by @iandvt in #4018
- llm: apply rate limit to all types by @howardjohn in #4029
- catalog: include all types by @howardjohn in #4016
- proxy: snapshot requests answered locally without an upstream call by @howardjohn in #3996
- Add
agctl bug-reportcommand by @howardjohn in #3991 - llm: expose more CEL attributes in the request processing phase by @howardjohn in #4010
- shrink make backend call stack by @howardjohn in #4035
- build(deps): bump the cargo-weekly group with 13 updates by @dependabot[bot] in #4045
- Update model catalog by @github-actions[bot] in #4038
- fix(ui): complete field-error hook dependencies by @iandvt in #4036
- fix(ui): use native elements for segmented controls and log rows by @iandvt in #4037
- ui: properly show schema help on all dropdowns by @howardjohn in #4034
- feat(controller): allow HTTPS and TLS listeners to share a port by @jimassa in #4000
- fix(mcp): reject invalid hosts in host/port/path configuration by @vladislav-miroshnikov in #4039
- Prepare for rust 1.100 by @howardjohn in #4046
- fix(llm): accept reasoning in completions-to-messages responses by @vladislav-miroshnikov in #4030
- agctl: add a new 'standalone' CLI by @howardjohn in #4032
- Benchmark and optimize krt collections by @howardjohn in #4017
- dynamic: deny destinations resolving to loopback by @howardjohn in #3889
- (breaking) standalone: always use a session key by @howardjohn in #4031
- (breaking) guardrails: support scope on vendor guardrails by @howardjohn in #3975
- fix(ui): correct custom provider endpoint hints by @pqminh27 in #4051
- Get substrate implementation up to date by @keithmattix in #4048
- tests: fix flaky auto protocol TCP tests by reading the full echo by @markuskobler in #4052
- Do not expose session.key over REST API by @howardjohn in #4056
- mcp: rebind re-used session to current settings by @howardjohn in #4057
- Add status check for MCP conditional CEL by @howardjohn in #4060
- ui: add a Continue recipe to Client Setup by @TheSeydiCharyyev in #4020
- refactor: unify cache implementation across policies by @howardjohn in #4047
- mcp: validate conditions match on each request, not just init by @howardjohn in #4062
- Add additional fields to analytics report by @howardjohn in #4061
- build: pin Go toolchain in go.mod + go 1.27.3 by @markuskobler in #4063
- llm: fix regression in DB logging errors by @howardjohn in #4071
- Allow opt-in controller admin bind address by @jbohanon in #4049
- llm: account for Anthropic usage iterations by @howardjohn in #4050
- mcp: advertise only change notifications upstreams offer by @howardjohn in #4054
- fix: allow parsing bracketted IPv6 by @howardjohn in #4073
- feat(ext_proc): accept hostname destinations from the endpoint picker by @ScraperNerd18 in #4012
- mcp: assert that resumed sessions are on the same backend by @howardjohn in #4059
- fix(ui): wait for LLM CORS before enabling Send by @pqminh27 in #4065
- feat(oidc): make sessionTtl configurable by @pqminh27 in #4066
- Align retry docs with reality by @howardjohn in #4077
- Drop deprecated AGENTGATEWAY_MESSAGES_PREFER_COMPLETIONS by @howardjohn in #4072
- drop unavailable lint by @howardjohn in #4074
- feat(helm): add extraInitContainers value to controller and standalone charts by @zackwy in #4053
- fix(ui): render log Markdown from a sanitized DOM fragment by @iandvt in #4040
- fix(ui): use native subpath imports in mergeEdits tests by @iandvt in #4033
- llm: read message_start usage in detect streams by @liorbenita2 in #4067
- llm: add constructors for common response types by @markuskobler in #4075
- fix(llm): forward OpenAI service_tier to Bedrock serviceTier by @matheus-meneses in #4078
- llm: revert Bedrock changes and drop messages preference env by @howardjohn in #4080
- feat(import): support LiteLLM default_fallbacks and define behavior for conditional fallbacks by @Altamimi-Dev in #3038
- examples: add Kubernetes and standalone response caching by @danehans in #3594
- refactor: cleanup rate limiting LLM specific code by @howardjohn in #4081
- examples: simplify NetBird integration by @danehans in #3942
- Error instead of panic on invalid SNI by @howardjohn in #4082
- Use native roots and AWS-LC for PostgreSQL TLS by @markuskobler in #4089
- Bump rust depends by @howardjohn in #4087
- csrf: treat Origin: null as a cross-origin request by @howardjohn in #4090
- proxy: do not leak error info to clients by @howardjohn in #2162
- Fix invalid assume-role session tags blocking proxy readiness by @lcopi in #4086
- fix: bedrock invalid function call recover by @conradj3 in #4027
- standalone: allow writing more MCP settings into DB by @howardjohn in #4094
- catalog: include deprecated models by @howardjohn in #4096
- fix inconsistent promptCache implementation by @lcopi in #3927
- csrf: compare only the Origin host, ignoring scheme by @howardjohn in #4091
- fix(controller): keep conflicted listeners out of traffic by @jbohanon in #3754
- aws: initial native endpoint support with runtime by @howardjohn in #3845
- Support openai 12h cache prompt by @howardjohn in #4088
- filters: better error handling by @howardjohn in #4097
- Fix redirect to another sceheme with explicit port by @howardjohn in #4092
- fix(mcp): treat 404/405 on upstream session DELETE as success by @yavzius in #4015
- fix(llm): keep schema formats in Gemini requests by @danielle-mustillo in #4095
- feat(llm): convert OpenAI inline file parts to Anthropic documents by @Yoel-qin in #3912
- Bump npm deps by @howardjohn in #4099
- fix(mcp): distinguish authorization denials in logs by @shubhamojha1 in #4079
- proxy: fix and test re-entrant locking by @howardjohn in #4105
- agctl: drop required apiVersion on exec auth by @howardjohn in #4102
- docs: add note about advisor tools by @howardjohn in #4101
- Move slow integration test to a unit test by @howardjohn in #4104
- Update model catalog by @github-actions[bot] in #4110
- fix(llm): preserve Gemini tool-call thought signatures by @markuskobler in #3898
- build: update Node to 24.22.1 by @Charlesthebird in #4113
- Use Substrate v1 gRPC service names by @EItanya in #4124
- omit empty bedrock tool list for completions by @stevenctl in #4107
- fix empty Bedrock tool arguments in streaming responses by @stevenctl in #4106
- fix mcp provider metadata to honor the configured AS url by @stevenctl in #4125
- fix: preserve CONNECT authority for Substrate ingress by @EItanya in #4111
- llm: better Detect mode by @howardjohn in #4136
- Add support for /v1/audio/* endpoints by @howardjohn in #4152
- build: update golangci-lint to v2.15.0 and kube-api-linter by @dongjiang1989 in #4123
- build(deps): bump tokio from 1.54.0 to 1.54.1 by @dependabot[bot] in #4154
- ui: do not default to mantle for bedrock by @howardjohn in #4155
- build(deps): bump rstest from 0.27.0 to 0.28.0 in the cargo-weekly group by @dependabot[bot] in #4150
- llm: improvements to stream_options by @howardjohn in #4156
- build(deps): bump tokio from 1.54.0 to 1.54.1 in /fuzz in the cargo group across 1 directory by @dependabot[bot] in #4153
- config: reload when file-backed auth secrets change by @stevenctl in #4109
- InferencePool allow policy attachment by @howardjohn in #4157
- validate MCP id in tool call by @howardjohn in #4158
- ext_authz, ratelimit: don't log header values from check responses by @vivek-raj1 in #4139
- Bump gateway api to v1.7.0-rc3 by @howardjohn in #4147
- feat(import): map LiteLLM cooldown settings to model health eviction by @FenjuFu in #3179
- ui: preserve comments in raw config editor by @jenshu in #4135
- Remove InterceptDenied and expose substrate specific error messagss by @keithmattix in #4093
- standalone: only expand env vars in API keys by @howardjohn in #4163
- llm: preserve content in redacted reasoning by @howardjohn in #4164
- agentgateway model: better respect allowedroutes config by @howardjohn in #4168
- catalog: add openai audio models by @howardjohn in #4171
- mcp: validate duplicate target names by @howardjohn in #4160
- retry: respect explicit attempts: 0 on AgentgatewayPolicy by @howardjohn in #4180
- substrate: per-provider credential cache TTL and FetchSecret timeout by @codyhartsook in #4170
New Contributors
- @maximumG made their first contribution in #3797
- @choyongho-bart0401 made their first contribution in #3812
- @dev-ahuja-gl made their first contribution in #3776
- @uc4w6c made their first contribution in #3810
- @noony made their first contribution in #3817
- @conradhanson made their first contribution in #3869
- @lareeth made their first contribution in #3871
- @erentorlak made their first contribution in #3919
- @chrisgavin made their first contribution in #3878
- @johnsto made their first contribution in #3676
- @jimassa made their first contribution in #4000
- @vladislav-miroshnikov made their first contribution in #4039
- @pqminh27 made their first contribution in #4051
- @ScraperNerd18 made their first contribution in #4012
- @matheus-meneses made their first contribution in #4078
- @yavzius made their first contribution in #4015
- @danielle-mustillo made their first contribution in #4095
- @vivek-raj1 made their first contribution in #4139
- @codyhartsook made their first contribution in #4170
Full Changelog: v1.6.0...v1.7.0
AI summary LLM routing and Kubernetes API now more resilient
🎉 Welcome to the 1.6.0 release of the agentgateway project!
This release makes LLM routing more resilient, turns on the Kubernetes AgentgatewayModel API by default, adds zero-configuration cost tracking for common public models, improves OIDC and JWT authentication, and fills important MCP compatibility gaps. It also adds session affinity, per-key local rate limits, graceful draining, and richer OpenTelemetry output.
Before upgrading, review the breaking and behavior changes below. In particular, Anthropic Messages requests now prefer the OpenAI Responses format and provider base URLs without a path now use /.
Artifacts
Docker images are available:
cr.agentgateway.dev/agentgateway:v1.6.0cr.agentgateway.dev/controller:v1.6.0
Helm charts are available:
cr.agentgateway.dev/charts/agentgateway:v1.6.0cr.agentgateway.dev/charts/agentgateway-crds:v1.6.0cr.agentgateway.dev/charts/agentgateway-standalone:v1.6.0
Binaries are available below.
Quick Start
Follow the Kubernetes or Standalone quick start guide to get started.
🔥 Breaking changes
Anthropic Messages requests now prefer the Responses format
Agentgateway now prefers OpenAI Responses when it converts Anthropic Messages requests. In 1.5, it preferred Chat Completions. This affects OpenAI, Ollama, Groq, Hugging Face, and xAI providers; Azure models other than Claude; and custom providers that advertise both Responses and Chat Completions.
The Responses conversion drops extended-thinking history, while the Chat Completions conversion preserves it. Use the OpenAI provider only for the OpenAI API. For other OpenAI-compatible servers, configure a custom provider and advertise only the formats that the server supports. Advertise only Chat Completions if the server does not implement /v1/responses or clients require extended-thinking history.
As a temporary workaround for a Responses conversion bug, set AGENTGATEWAY_MESSAGES_PREFER_COMPLETIONS=true. This compatibility setting is planned for removal in 1.7.
A built-in model catalog now supplies default request pricing
Requests to common public models now have a cost in logs, traces, metrics, and CEL without a separately configured catalog. As a result, USD API-key budgets in standalone and cost-based policies or CEL expressions in either deployment mode can begin applying to these models after the upgrade.
Review USD budgets and expressions that use llm.cost. Existing catalog configuration can be removed or retained as an overlay to customize the built-in rates.
Provider base URLs without a path now use /
An explicitly configured provider base URL now always sets the complete base path. A URL such as https://api.openai.com therefore uses /. In 1.5, custom and Ollama providers appended /v1, while built-in providers such as OpenAI forwarded the client's path.
Add the provider API path to every configured base URL that currently has no path. For example, change https://api.openai.com to https://api.openai.com/v1 and http://localhost:11434 to http://localhost:11434/v1. URLs that already contain a path and providers that use their default address are unaffected.
Standalone LLM serving paths must match exactly
Standalone's simplified llm configuration now recognizes standard LLM paths only by exact match. A prefixed path such as /tenant-a/v1/messages is passed through without format conversion unless you configure llm.pathPrefix, such as pathPrefix: /tenant-a.
On Kubernetes, model routers attached directly to a Gateway or ListenerSet likewise serve only exact standard LLM paths. Attach models to an HTTPRoute to serve them under a prefix.
The standalone Helm chart requires oidc.enabled
The standalone chart now sets OIDC_COOKIE_SECRET only when the new oidc.enabled value is true; the value defaults to false. If you use oidc.cookieSecretName to protect the UI or another app, set oidc.enabled=true before upgrading or agentgateway will reject the OIDC configuration at startup.
⚠️ Removed
- Legacy token-count semantics:
AGENTGATEWAY_LEGACY_LLM_USAGE_TOKEN_SEMANTICShas been removed. Input and total token counts always include prompt-cache tokens. - Server-side defaults in Kubernetes CRDs: CRD schemas no longer persist default values such as
action: Allowortracing.protocol: GRPC. The controller applies the same defaults at runtime, so traffic behavior does not change, butkubectl get -o yamldisplays only explicitly configured fields. Update GitOps comparisons and scripts that expected stored defaults.
🔄 Other behavior changes
- Failover eviction: A backend or virtual model with multiple priority groups now evicts failing targets by default. Defining a health policy replaces this default, so include an eviction policy explicitly when you want to retain failover behavior.
- Streaming guardrails: A provider guard failure while checking a streamed response or realtime connection now rejects content. Set the guard's
failureModetofailOpento retain the 1.5 behavior. - Guardrail results: The
guardrailsCEL variable includes every guard that ran, including a newallowaction. Filter onaction != "allow"when you need interventions only. - Backend authentication errors: Failures while obtaining provider credentials, such as from OAuth, AWS STS, or Azure, return
502instead of500. Local credential application failures return500instead of503. - Backend request timeouts: Backend request timeouts now include the time required to read a buffered body, including bodies read by external authorization and CEL
response.bodyexpressions. Streamed bodies are unaffected. - Policy service timeouts: gRPC external authorization calls now default to 2 seconds; rate limit and external processing calls default to 10 seconds.
- Access-log levels: Request records use
errorwhen agentgateway records a request error andinfootherwise. Previously all request records usedinfo. - Standalone YAML handling: Unquoted values such as
noremain strings instead of becoming booleans. Usetrueorfalsefor boolean settings. Parse errors now include line numbers, and UI writes preserve comments and formatting where possible. - Standalone catalog refresh: The UI's Refresh base costs button downloads the catalog from the repository's
mainbranch. Do not use it with 1.5 or earlier ifmaincontains the 1.6 catalog format. - Kubernetes policy conflicts: When equally specific policies set the same field, the oldest policy by
metadata.creationTimestampwins consistently. - Kubernetes AI transformations:
spec.backend.ai.transformationsandspec.backend.ai.finalTransformationsreject duplicatefieldvalues. Existing policies with duplicates fail validation when reapplied. - Kubernetes deployer ownership: The controller no longer overwrites an existing same-named resource that it does not own and reports an error instead.
🌟 New features
LLM routing, formats, and cost controls
AgentgatewayModelenabled by default: The KubernetesagentgatewayModels.enabledHelm value now defaults totrue.- Wildcard model discovery:
/v1/modelsexpands wildcard model configurations into matching model IDs from the catalog. Standalone users can setllm.discovery: disabledto retain the 1.5 behavior of returning the wildcard pattern. - More resilient virtual models: Kubernetes virtual models skip broken concrete targets instead of failing the entire model, and backends with one invalid inline policy can remain available with
PartiallyValidstatus. - Response idle timeout: A new response idle timeout ends a response when the backend stops sending body data for the configured duration without limiting an active stream's total lifetime.
- Larger LLM request buffer: The default LLM request buffer increases from 2 MiB to 32 MiB for long-context prompts.
- Per-page pricing: Model catalog entries support
rates.perPagefor OCR and document models. - Bedrock Mantle routing: Bedrock can choose the Runtime or Mantle endpoint per model, with a configurable endpoint preference.
- More complete Messages conversion: Conversion between Anthropic Messages and OpenAI formats now carries citations, refusals, strict tool schemas, reasoning effort, and images in tool results. Provider context-overflow errors are translated so Claude Code can compact and retry.
- Guardrail controls: OpenAI Moderation, Bedrock Guardrails, and Google Model Armor support
failureMode. Standalone webhook guards can also configure connection policies, and the built-in credit-card detector now validates the Luhn checksum.
Security and identity
- OIDC sign-in for standalone: OIDC policies can expose login and logout endpoints. Unauthenticated
fetchrequests receive401instead of an identity-provider redirect, and compressed session cookies accommodate users with more group claims. The UI configures the endpoints automatically. - Hashed API keys: The standalone UI stores newly created API keys as hashes by default.
- Smarter JWT provider selection: When several JWT providers are configured, agentgateway tries providers whose issuer and JWKS key ID match the token. JWT validation also enforces
nbfwith 60 seconds of clock-skew leeway. Kubernetes policies can configurevalidation.requiredClaims; standalone MCP authentication no longer requires anaudiencesfield. - Cloud backend authentication: AWS assume-role authentication accepts an
externalId, and Azure authentication accepts explicitscopes. - Network authorization by destination: Network authorization expressions can match
destination.address,destination.port, and the TLS SNI value indestination.hostname. - Kubernetes CA bundle keys: Backend TLS can read a CA bundle from a Secret or ConfigMap key other than
ca.crt, including keys written by trust-manager.
MCP
- Paginated lists: Tool, prompt, and resource list responses include
nextCursor. Federated endpoints return a combined cursor that tracks all targets. - MCP data in CEL: Policies can match
mcp.methodName, such astools/call, and standalone access logs can record list results such asmcp.toolsList. - Server information overrides: Standalone multiplexed gateways can override reported
serverInfoand instructions with anmcp.serverblock. - SSE keep-alive: Standalone can send periodic comments on long-lived MCP streams so idle intermediaries do not close them.
- Request size enforcement: MCP requests larger than the configured buffer limit return
413.
Traffic management and operations
- Session affinity on Kubernetes: A CEL expression can derive a session value and consistently send related requests to the same endpoint.
- Per-key local rate limits: CEL-derived keys give each user, API key, or other value a separate local rate-limit bucket.
- External processing fail-open: Standalone routes can forward a request when the external processing service fails, including the request body when no body bytes have yet been sent to the processor.
- Graceful standalone drain: Shutdown can stop accepting new connections at a minimum termination deadline while allowing existing connections to continue until the final termination deadline.
- Standalone autoscaling and disruption budgets: The standalone Helm chart can create a HorizontalPodAutoscaler and PodDisruptionBudget.
- Stable ListenerSet ordering: Kubernetes ListenerSets sort by creation time, oldest first, then namespace and name.
- Reduced Istio permissions: Set
istio.enabled=falseto remove Istio permissions from the Kubernetes controller ClusterRole. - Gateway-aware proxy metrics: The Kubernetes PodMonitor copies the Gateway name pod label onto proxy metrics by default, enabling per-Gateway dashboard filtering.
- OpenTelemetry updates: OTLP access logs use the
agentgateway.accessinstrumentation scope, and failed requests carry anerror_typelabel on request-duration metrics. Stdout access logs can opt in to OpenTelemetry field names with the OTel access-log preset.
Contributors
Thank you to everyone who contributed code, reviews, documentation, bug reports, and CI improvements for this release!
What's Changed
- bedrock: default region even when using host override by @howardjohn in #3207
- Fix stack overflow in debug builds with UI writes by @howardjohn in #3209
- pool: max connection duration by @stevenctl in #3210
- Remove stale comments by @howardjohn in #3213
- helm: allow disabling istio permissions by @howardjohn in #3218
- test: filter golden tests to only relevant fields by @howardjohn in #3219
- standalone: allow disabling audience validation for mcp auth by @howardjohn in #3216
- fix(controller): add InferencePool status re-merge in StatusSyncer like other resources by @koba1t in #3250
- docs: emit
*for fields that take arbitrary fields by @howardjohn in #3220 - cel: expose
proxy.errorattributes by @howardjohn in #3215 - fix(llm): support Cohere Embed v4 responses by @jstar0 in #3244
- api: reject duplicate fields in AI transformation by @howardjohn in #3253
- chore(ui): enable Biome linting by @iandvt in #2959
- mcp: populate
mcp.methodNameduring RBAC evaluation by @onematchfox in #3197 - llm: strip alternative auth keys for gemini by @howardjohn in #3255
- llm: preserve error on count tokens error by @howardjohn in #3254
- build(deps): bump the cargo-weekly group with 9 updates by @dependabot[bot] in #3248
- proxy: avoid recursive make_backend_call by @howardjohn in #3256
- deployer: do not take over resources by @howardjohn in #3221
- ui: warn when API key budgets lack database by @charles-chenzz in #3222
- make: allow weird go versions by @howardjohn in #3257
- catalog: check in catalog, embed in binary, and automate updates by @howardjohn in #3191
- telemetry: derive access-log severity from response status/error by @ETretyakov in #3112
- Update model catalog by @github-actions[bot] in #3258
- fix: add HTTP/2 PING keepalive for upstream connection pool by @marwanad in #3231
- fix(ext-proc): preserve response trailers by @Aias00 in #3226
- Dockerfile: copy catalog for builds by @nfuden in #3266
- crypto: add optional AWS-LC-FIPS backend by @ymesika in #3225
- ci: avoid no-op changes to catalog by @howardjohn in #3269
- Fix OTLP status for provider HTTP errors by @danehans in #3261
- Add affinity to Kubernetes by @howardjohn in #3268
- test(e2e): make multicontroller fixture configurable by @danehans in #3217
- config: warn more loudly about deprecated logging config by @howardjohn in #3271
- fix(telemetry): align HTTP attributes with OTel semconv by @harukz in #3182
- gw-api: Support XBackend by @keithmattix in #2634
- fix(ui): support OpenAPI targets in the MCP Servers editor by @Altamimi-Dev in #3039
- llm: rewrite multipart model fields for routed models by @Geun-Oh in #3131
- catalog: use github as source of truth by @howardjohn in #3275
- ui: no limit on dropdown items by @howardjohn in #3273
- llm: handle adaptive thinking by @howardjohn in #3277
- substrate: authorize actor egress at CONNECT time by @EItanya in #3237
- ui: avoid rebuilding full project when ui changes by @howardjohn in #3279
- ui: embed generated schemas by @howardjohn in #3280
- pii: add luhn validation to credit card check by @howardjohn in #3270
- substrate: emit ate.router.resume so cold starts are visible by @krisztianfekete in #3289
- cargo: use string for rustflags by @howardjohn in #3292
- Update model catalog by @github-actions[bot] in #3286
- messages: report matched stop sequences by @npolshakova in #3283
- cel: fill in bytes addition gap by @howardjohn in #3297
- mod bump grpc in go by @nfuden in #3295
- Update model catalog by @github-actions[bot] in #3307
- Bump all golang dependencies by @howardjohn in #3303
- github: add PR and issue template around LLM usage by @howardjohn in #3300
- rl: improve debug logging and skip no-op requests entirely by @howardjohn in #3299
- fix(controller): propagate Gateway ancestry through virtual model failover targets by @dongjiang1989 in #3311
- add scopes to azureAuth by @jgreeer in #3317
- refactor: make listener protocol/TLS mapping reusable by @jbohanon in #3313
- Parse MCP context for CEL early by @howardjohn in #3301
- fix(resource_manager): reschedule failed HTTP refresh so retries actually fire by @loafoe in #3290
- helm: allow disabling the config checksum annotation by @cbascom in #3304
- proxy: invalid cached body on body mutations by @howardjohn in #3321
- Update model catalog by @github-actions[bot] in #3330
- substrate: emit ate.actor.uid and ate.router.route.duration by @krisztianfekete in #3333
- examples: add NetBird Agent Network integration by @danehans in #3111
- Update async-openai by @howardjohn in #3338
- examples: use official NetBird release images by @danehans in #3341
- extproc: Send EOS when there are no trailers by @keithmattix in #3336
- feat(llm): add timeout on LLM policy by @rasta-rocket in #3366
- fix: fall back to Host header for HTTP/2 requests missing :authority by @richistron in #3368
- fix(telemetry): zero-pad the sub-second part of log timestamps by @AhmadMasry in #3370
- auth: classify backend authentication failures by @shubhamojha1 in #3177
- bedrock guardrails: allow passthrough by default by @nfuden in #3276
- controller: add jwtValidationOptions to AgentgatewayPolicy by @a7i in #3381
- build(deps): bump the cargo-weekly group with 11 updates by @dependabot[bot] in #3367
- Adding new Example to deploy agentgateway standalone on render.com by @sebbycorp in #3361
- tracing: add parentNotSampled to sample unsampled parent by @krisztianfekete in #3316
- fix: enforce idle timeout for streaming responses by @fengxsong in #3310
- test: always setup logs on all test paths by @howardjohn in #3387
- messages: accept tool_reference and unknown tool_result parts by @pazshalev in #3349
- Fix rate-limit denial headers and add Retry-After by @howardjohn in #3386
- Update model catalog by @github-actions[bot] in #3344
- fix(mcp): propagate _meta envelope to multi-target resolve requests by @dongjiang1989 in #3372
- substrate: drop mutex on ingress path by @howardjohn in #3335
- fix: carry the finished text in Responses stream terminal events by @npolshakova in #3326
- gemini: support new api key format by @howardjohn in #3388
- Update model catalog by @github-actions[bot] in #3392
- Substrate Egress Policy Implementation by @keithmattix in #3318
- fix(3384): stable sorting and inclusion of parent CreationTimestamp on ListenerSets by @jbohanon in #3400
- cel: preserve order in maps by @howardjohn in #3398
- config: dynamically reload standardAttributes by @howardjohn in #3402
- mcp: classify InvalidRequest/Unavailable for modern notifications in handle_error by @ETretyakov in #3113
- fix: preserve trailing slash in ReplaceFullPath rewrite and redirect by @ymesika in #3396
- llm: always treat baseURL as the full base URL by @howardjohn in #3403
- AgentgatewayModel: avoid overrwriting default routes by @howardjohn in #3391
- Misc UI improvements by @howardjohn in #3407
- feat: configure gateway bind address by @shubhamojha1 in #3406
- llm: increase default buffer size to 32mb for LLM by @howardjohn in #3408
- Move to the new substrate ingress header by @keithmattix in #3409
- feat: import LiteLLM upstream request headers by @ZenAlexa in #3233
- policies: in conflict, consistently pick oldest policy by @howardjohn in #3294
- Add Datadog observability example by @danehans in #3259
- llm: partially degraded virtual models by @howardjohn in #3320
- fix(cel): wrong-typed receiver should not be treated like an absent one by @charles-dyfis-net in #2879
- fix: report AddResourceExtension fields contributed by two plugins by @jbohanon in #3312
- backend: accept partially invalid backend by @howardjohn in #3389
- Use default 80 for non-CONNECT ingress by @keithmattix in #3410
- Update model catalog by @github-actions[bot] in #3413
- fix(podMonitor): populate gateway labels for the Grafana dashboard by @symaras in #3421
- test: support configurable e2e core charts by @danehans in #3423
- fix(xds): harden leaf cert rotation with observability by @dongjiang1989 in #3170
- proxy: add default timeouts for missing policy calls by @howardjohn in #3426
- feat(mcp): add configurable SSE keep-alive for long-lived MCP streams by @TechPrototyper in #3393
- docs: repoint stale documentation URLs by @kristin-kronstain-brown in #3427
- ui: use playwright-managed chromium by @jenshu in #3422
- ci: fix broken blacksmith cache by @howardjohn in #3429
- Support itl by @gaogaoSpark in #2984
- substrate: Fix custom port by @keithmattix in #3428
- adding a standalone dashboard by @linsun in #3108
- feat(controller): select the CA bundle key in AgentgatewayPolicy CA refs by @BoxBoxJason in #3419
- fix(telemetry): stop overriding otlp scope by @krisztianfekete in #3430
- fix(llm): carry thinking blocks between Messages and Chat Completions by @ehfd in #3363
- Update model catalog by @github-actions[bot] in #3435
- fix(bedrock): parse encrypted redactedContent reasoning blocks in non-streaming responses by @laithalsaadoon in #3235
- llm: fix e2e tests by @howardjohn in #3441
- llm/bump async openai by @howardjohn in #3445
- mcp: drop debug log by @howardjohn in #3446
- Fix non-anthropic reasoning with bedrock by @howardjohn in #3444
- Update model catalog by @github-actions[bot] in #3448
- build(deps): bump the cargo-weekly group with 18 updates by @dependabot[bot] in #3454
- fix(llm): keep usage when the final stream chunk omits delta by @fengxsong in #3455
- llm: gracefully degrade responses websocket by @howardjohn in #3461
- New body representation by @howardjohn in #3432
- llm: automatically enable eviction for failover by @howardjohn in #3462
- Drop legacy llm token env var by @howardjohn in #3208
- substrate: egress credential injection by @keithmattix in #3411
- proxy: allow trailers in all cases by @howardjohn in #3463
- refactor: standardize sensitve header selection by @howardjohn in #3464
- ci: mark release as pre-release when appropriate by @howardjohn in #3465
- feat(http): key local rate limit buckets with CEL by @ehfd in #3351
- oidc: issue 401 instead of redirect for fetch by @howardjohn in #3281
- proxy: keep serving open connections during drain by @jlaneve in #3334
- refactor: utilize new body for easier json caching by @howardjohn in #3466
- llm: re-use cached JSON parse by @howardjohn in #3469
- Update model catalog by @github-actions[bot] in #3475
- promptGuard: give the webhook target the same
policiesslot as extAuthz (backendTLS) by @zach-weisner in #3474 - bedrock mantle/runtime catalog routing by @nfuden in #3187
- Fix test flake in new eviction test by @howardjohn in #3467
- perf: do not clone llm completion by @howardjohn in #3468
- bedrock: support namespaced tools from responses by @howardjohn in #3447
- feat: add OpenID AuthZEN authorization example for MCP by @embesozzi in #3478
- Fix merge conflict by @howardjohn in #3488
- Fix GIE dependency bump automation by @howardjohn in #3252
- feat(mcp): allow overriding gateway serverInfo and instructions by @alexoakland in #3425
- security: add CRA docs by @howardjohn in #3484
- controller: drop the resourceVersion from the status write queue key by @chizhang21 in #3480
- jwt: validate NBF claim by @howardjohn in #3486
- Fix improper messages->responses id and prepare golden files by @howardjohn in #3491
- AgentgatewayModel: fix a few collisions around multiple listeners and sectionName by @howardjohn in #3489
- perf: offload log serialization to log thread by @howardjohn in #3470
- Messages to Responses: fix openai moderation guardrail by @howardjohn in #3493
- Use resolved LLM models throughout request processing by @howardjohn in #3481
- messages to responses: various compatibility fixes by @howardjohn in #3496
- llm: messages->completions support effort without adaptive thinking by @howardjohn in #3497
- ui: add support for bedrock mantle selection by @howardjohn in #3487
- Apply response idle timeout to the original upstream body by @howardjohn in #3495
- Messages to Responses: preserve images in tool responses by @howardjohn in #3498
- chore(ui): enable recommended biome linting by @iandvt in #3305
- ui: drop new flakey test by @howardjohn in #3508
- feat: allow to setup PodDisruptionBudget in agw standalone helm chart by @rsicart in #3503
- Update model catalog by @github-actions[bot] in #3499
- CONNECT: skip explicit listener bind by @howardjohn in #3494
- feat(aws): add support for externalId in assumeRole by @gordon-to in #3449
- llm: properly convert 'strict' schema field by @howardjohn in #3509
- llm: fix double refusal emission by @howardjohn in #3510
- llm: use shared partial tool parsing logic on messages->responses by @howardjohn in #3512
- llm: map messages content_filter to openai refusal by @howardjohn in #3514
- ui/oidc: Add explicit login and logout by @howardjohn in #3483
- llm: messages to responses citations by @howardjohn in #3515
- llm: messages to responses better cache control by @howardjohn in #3516
- llm: messages to responses record ITL by @howardjohn in #3517
- llm: add missing Gemini schema fields by @matvas-txf in #3501
- substrate: align credential injection with provider contract by @EItanya in #3524
- examples: add JEV guardrail example by @howardjohn in #3529
- ui: add Pi harness setup guide by @howardjohn in #3531
- ui: do not hide guardrail rejection logs by @howardjohn in #3530
- llm: Better messages to openai effort/thinking by @howardjohn in #3518
- standalone: trim 100++ line of startup logs by @howardjohn in #3532
- feat: allow to setup HorizontalPodAutoscaler in agw standalone helm chart by @rsicart in #3521
- Update model catalog by @github-actions[bot] in #3522
- ui: fix prompt log message by @howardjohn in #3535
- ui: avoid overly pruning fields where empty has meaning by @howardjohn in #3534
- ui: show confirmation when navigating away from guardrail page with pending edits by @howardjohn in #3536
- build(deps): bump oras.land/oras-go/v2 from 2.6.1 to 2.6.2 in /tools in the go_modules group across 1 directory by @dependabot[bot] in #3537
- ui: show guardrail result in logs by @howardjohn in #3533
- transformation: simplify deserialization by @howardjohn in #3538
- iterate on mcp errors from gateway on tool call response by @nfuden in #3526
- feat: price non-token billing per page for ocr workflows by @LaloVene in #3395
- expose destination (SNI) attribute in network authz by @howardjohn in #3540
- gie: move to v1.6.2 off our fork by @howardjohn in #3541
- Link fixes by @kristin-kronstain-brown in #3548
- Update model catalog by @github-actions[bot] in #3547
- syncer: add WithExtraListenerSets and WithAllowedListenersResolver by @jbohanon in #3460
- build: use -trimpath by @howardjohn in #3554
- llm: check if format supports guards before attempting response guard by @howardjohn in #3555
- oidc: compress the browser session cookie payload by @jbohanon in #3502
- guardrails: optimize memcpy around streaming by @howardjohn in #3558
- controller: share common Istio scheme by @howardjohn in #3559
- perf(controller): cache TLS validation results by @shubhamojha1 in #3458
- Fail closed per-backend for invalid GCP credentials by @jenshu in #3560
- Cache CEL syntax check by @howardjohn in #3561
- build: disable cruft from Istio/envoy by @howardjohn in #3556
- controller: ListenerSet does not depend on Gateway status by @howardjohn in #3562
- controller: skip rate set allocation by @howardjohn in #3563
- deps: bump rust dependencies by @howardjohn in #3543
- controller: fix krt key collision by @howardjohn in #3567
- build(deps): bump go.opentelemetry.io/otel/sdk from 1.44.0 to 1.46.0 in the go_modules group across 1 directory by @dependabot[bot] in #3565
- controller: optimize listener sets in gateway by @howardjohn in #3564
- feat(helm): allow annotations on the Grafana dashboard ConfigMap by @QuentinBisson in #3591
- standalone: require explicit Helm opt in for OIDC by @howardjohn in #3596
- Enable AgentgatewayModel by default by @howardjohn in #3492
- ui: show models from the config dump in XDS mode by @TheSeydiCharyyev in #3570
- fix(mcp): return 413 for oversized request bodies by @DEOWL-kan in #3593
- fix(llm): preserve optional fields in structured output schemas by @iandvt in #3577
- Update model catalog by @github-actions[bot] in #3576
- Bump deps for UI by @howardjohn in #3600
- fix(ui): remove non-null assertions by @iandvt in #3582
- controller: use MapColletion in policies by @howardjohn in #3568
- api: stop server-side defaulting by @howardjohn in #3520
- ext proc: handle fail open with body by @howardjohn in #3542
- Bump go mod deps by @howardjohn in #3614
- fix(jwt): select providers by issuer and kid by @justinmpowers in #3611
- Update model catalog by @github-actions[bot] in #3606
- fix(mcp): harden OpenAPI path parameter handling by @markuskobler in #3598
- ui: when an API key is provisioned, stamp the request identity into it by @howardjohn in #3620
- Record GenAI request duration failures by @danehans in #3319
- (breaking) llm: only match exact path instead of any suffix by @howardjohn in #3539
- istio: respect revision in ca address by @howardjohn in #3625
- llm: avoid null cache_write_tokens on responses by @howardjohn in #3619
- Prevent xDS credentials from reaching trace logs by @DevArjun23 in #3624
- guardrails: add fail open/fail closed polciy to all types by @howardjohn in #3618
- AgentgatewayModel: fix gateway ancestry by @howardjohn in #3617
- controller: short circuit expensive listener sort by @howardjohn in #3627
- tracing: drain body in proper runtime by @howardjohn in #3631
- conformance: do not skip static address by @howardjohn in #3638
- Update model catalog by @github-actions[bot] in #3636
- fix(llm): translate context overflow errors for Claude Code recovery by @iandvt in #3581
- feat: expose the backend endpoint in backend cel context by @fengxsong in #3633
- Support cursors in list in MCP by @howardjohn in #3544
- controller: avoid double resolution of jwks by @howardjohn in #3628
- mcp: expose
list/*results into CEL by @howardjohn in #3601 - controller: partially fetch lsitenerset by @howardjohn in #3641
- Adds a tier-aware semantic routing example that uses one vSR runtime by @danehans in #3471
- xds: log more accurate size of payload by @howardjohn in #3643
- proxy: shrink XDS storage after resources go away by @howardjohn in #3642
- controller: optimize listener conflcit with pre-allocation by @howardjohn in #3644
- config: replace YAML processing library by @howardjohn in #3639
- ui: better embedded catalog support by @howardjohn in #3645
- Better support for Jev by @howardjohn in #3646
- llm: stop on error by @howardjohn in #3511
- Add support for conditional Virtual MCP Targets by @keithmattix in #3615
- Update model catalog by @github-actions[bot] in #3654
- bump dep and clean up osv-scanner noise by @markuskobler in #3658
- Fix ext_proc clean stream close handling by @keithmattix in #3648
- llm: Expand models in /v1/models by @howardjohn in #3649
- llm: preserve additionalProperties in Gemini schema normalization by @matvas-txf in #3590
- CI: add nice diff for model catalog PR by @howardjohn in #3660
- timeout: respect backend request timeout for any body buffering by @howardjohn in #3599
- bedrock mantle: validate guardrail support by @howardjohn in #3507
- (breaking) llm: default messages -> responses (was completions) by @howardjohn in #3647
- catalog: future proof unknown fields by @howardjohn in #3669
- Misc UI cleanup by @howardjohn in #3672
- Update model catalog by @github-actions[bot] in #3675
- Fix Substrate egress after the ateom/actor identity split by @EItanya in #3677
- Bump istio dependency by @howardjohn in #3678
- ui: support and default to hashed keys by @howardjohn in #3671
- ui: misc minor cleanup by @howardjohn in #3685
- Update model catalog by @github-actions[bot] in #3680
- Keep service workload endpoints out of backend CEL context by @howardjohn in #3679
- build(deps): bump the cargo-weekly group with 9 updates by @dependabot[bot] in #3684
- fix(transformation): leave the upstream body when a CEL body expression is null by @conradj3 in #3653
- Fix jwks direct URI nil pointer by @howardjohn in #3699
- gateway api: future proof 1.7 by @howardjohn in #3697
- ci: resolve e2e warm-test toolexec path from repo root by @Jpatel86 in #3704
- fix gemini cost catalog lookup by @stevenctl in #3709
- [fix] pdb standalone Helm installation maxUnavailable precedence by @artberger in #3720
- Update model catalog by @github-actions[bot] in #3693
- include provider in request parsing errors by @stevenctl in #3712
- fix: resolve bare backend references in local config by @Yoel-qin in #3690
- max_connection_age: add jitter setting for frontend and backend by @stevenctl in #3214
- llm: fix all vertex catalog lookups by @howardjohn in #3726
- misc CI optimizations by @howardjohn in #3725
- kubernetes: expose backend authorization policy by @howardjohn in #3713
- proxy: fix regex matching by @howardjohn in #3724
- Fix GRPCRoute method matches by @howardjohn in #3723
- build(deps): bump the uv group across 1 directory with 2 updates by @dependabot[bot] in #3728
- ui: cleanup LLM defaults by @howardjohn in #3688
- ci: make release a bit faster by @howardjohn in #3730
- ui: add version on sidebar by @howardjohn in #3733
- llm: translate token usage to format-specific format by @howardjohn in #3687
- kubernetes: expand wildcards in model discovery by @howardjohn in #3731
- regex guardrails: make matches not greedy past the actual match by @howardjohn in #3735
- oidc: add initial refresh flow support by @howardjohn in #3291
- build(deps): bump dompurify from 3.4.13 to 3.4.16 in /ui in the npm_and_yarn group across 1 directory by @dependabot[bot] in #3727
- fix(mcp): expose mcpGuardrails dynamic metadata to access-log CEL by @Jpatel86 in #3703
- substrate: fix egress cert rotation on long-lived tunnels and trace egress gRPC calls by @keithmattix in #3686
- llm: add meta provider by @howardjohn in #3741
- bump async-openai dependency by @howardjohn in #3736
- fix(mcp): return OpenAPI image responses as MCP image content by @lewis-treacy in #3722
- bump rust by @howardjohn in #3745
- Bump serde-yaml-edit dep by @howardjohn in #3742
- Implement full protocol selection from the Substrate policy by @EItanya in #3689
- pprof-alloc bump by @howardjohn in #3747
- llm: strip query param when translating to another format by @howardjohn in #3743
- gateway api: support new backend url rewrite filter by @howardjohn in #3755
- llm: future proof openai decisions endpoint by @howardjohn in #3756
- Update model catalog by @github-actions[bot] in #3750
New Contributors
- @charles-chenzz made their first contribution in #3222
- @ETretyakov made their first contribution in #3112
- @github-actions[bot] made their first contribution in #3258
- @marwanad made their first contribution in #3231
- @Aias00 made their first contribution in #3226
- @loafoe made their first contribution in #3290
- @rasta-rocket made their first contribution in #3366
- @richistron made their first contribution in #3368
- @AhmadMasry made their first contribution in #3370
- @a7i made their first contribution in #3381
- @sebbycorp made their first contribution in #3361
- @pazshalev made their first contribution in #3349
- @ZenAlexa made their first contribution in #3233
- @symaras made their first contribution in #3421
- @TechPrototyper made their first contribution in #3393
- @BoxBoxJason made their first contribution in #3419
- @ehfd made their first contribution in #3363
- @jlaneve made their first contribution in #3334
- @zach-weisner made their first contribution in #3474
- @embesozzi made their first contribution in #3478
- @alexoakland made their first contribution in #3425
- @chizhang21 made their first contribution in #3480
- @rsicart made their first contribution in #3503
- @matvas-txf made their first contribution in #3501
- @LaloVene made their first contribution in #3395
- @QuentinBisson made their first contribution in #3591
- @TheSeydiCharyyev made their first contribution in #3570
- @DEOWL-kan made their first contribution in #3593
- @justinmpowers made their first contribution in #3611
- @DevArjun23 made their first contribution in #3624
- @conradj3 made their first contribution in #3653
- @Yoel-qin made their first contribution in #3690
- @lewis-treacy made their first contribution in #3722
Full Changelog: v1.5.0...v1.6.0
AI summary Expanded LLM budgets, Gemini APIs, and enhanced proxy support
🎉 Welcome to the 1.5.0 release of the agentgateway project!
This release expands agentgateway across LLM, agent, and general-purpose traffic. Highlights include API-key-scoped LLM budgets, native Gemini inbound APIs, Anthropic Messages to OpenAI Responses conversion, deeper Agent Substrate integration, SPIFFE Workload API identities, and richer dynamic forward proxy and TCP support. Guardrails can now cover tool calls and use OpenAI inline moderation, while the UI, traces, logs, metrics, and agctl provide a much clearer view of multi-turn agent activity and policy callouts.
Artifacts
Docker images are available:
cr.agentgateway.dev/agentgateway:v1.5.0cr.agentgateway.dev/controller:v1.5.0
Helm charts are available:
cr.agentgateway.dev/charts/agentgateway:v1.5.0cr.agentgateway.dev/charts/agentgateway-crds:v1.5.0cr.agentgateway.dev/charts/agentgateway-standalone:v1.5.0
Binaries are available below.
Quick Start
Follow the Kubernetes or Standalone quick start guide to get started.
🔥 Breaking changes
LLM token counts now include prompt-cache tokens
Agentgateway now normalizes LLM usage across providers: llm.inputTokens includes cache-read and cache-creation tokens, and llm.totalTokens is that normalized input count plus output tokens. The new llm.providerInputTokens and llm.providerTotalTokens fields preserve the provider's original values. This changes access logs, spans, metrics, token-based rate limits, and CEL expressions for Anthropic, Bedrock, and other providers that exclude cached tokens from their reported input count; cost calculation is unchanged.
Review token-based limits and dashboards before upgrading. As a temporary migration aid, set AGENTGATEWAY_LEGACY_LLM_USAGE_TOKEN_SEMANTICS=true to restore the previous behavior. This compatibility setting is planned for removal after 1.5.
JWT issuer and audience claims are now required when configured
JWT authentication now requires an iss claim that matches the configured issuer. When a non-empty audience list is configured, the token must also contain a matching aud claim. Setting requiredClaims to an empty list no longer disables these requirements; it only removes additional claim requirements beyond those implied by the issuer and audiences.
Review JWTs issued to clients before upgrading, especially tokens that previously omitted iss or aud. For more information, see the Kubernetes JWT authentication and Standalone JWT authentication docs.
Backend and attached LLM policies now merge field by field
An AI policy defined directly on an AgentgatewayBackend now merges field by field with an attached AgentgatewayPolicy. Previously, the backend policy replaced the complete attached AI policy block. This fixes attached prompt guards, defaults, transformations, model aliases, and prompt caching being silently dropped, but configurations that relied on full replacement must be updated.
Cross-namespace route delegation requires a ReferenceGrant
Cross-namespace Route-to-Route delegation now requires a ReferenceGrant from the child route's namespace. Add the grant before upgrading or the delegation will no longer be accepted. For more information, see route delegation and GHSA-g7j8-wp7h-wmgr.
Managed API key metadata uses the agentgateway.dev/ prefix
For standalone API keys created through the admin UI or API, the key ID moved from metadata.id to metadata["agentgateway.dev/id"], and a new metadata["agentgateway.dev/createdAt"] field records creation time. User-supplied metadata under the reserved agentgateway.dev/ prefix is now rejected. Update integrations that read or write these fields.
Legacy Istio identity TLV support has been removed
Agentgateway no longer emits the special Istio identity TLV that was intended for a ztunnel "sidecar". Use agentgateway's native mTLS support for workload identity. Deployments that still require the sandwich topology can continue to pass traffic, but no longer receive identity through this TLV.
🌟 New features
API-key-scoped budgets and model access
Standalone deployments can assign rolling token or dollar budgets to individual API keys. A budget can block requests or audit an overage, and its state is persisted to SQLite or PostgreSQL and can be viewed and managed in the UI. API keys can also restrict which models a caller is authorized to use. Budgets require hybrid storage mode.
Better LLM log views
LLM request logs now include in-depth request and response details, such as tool calls, reasoning, token usage, and request trajectory.
Native Gemini APIs and broader format conversion
Clients using Gemini and Vertex SDKs can now call agentgateway through the native generateContent, streaming streamGenerateContent, and countTokens APIs.
Agentgateway can also translate Anthropic Messages requests to OpenAI Responses backends, including buffered and streaming text, images, function tools, structured output, usage, finish reasons, and error mapping. This makes Responses-only providers available to clients that speak the Anthropic Messages API. For provider configuration, see the Kubernetes and Standalone LLM provider docs.
Additional LLM improvements include:
- Amazon Nova multimodal embeddings on Bedrock and Cohere v4 embeddings on Bedrock.
- OpenAI Responses routing for GitHub Copilot Grok models.
- Translation of OpenAI prompt-cache markers to Anthropic and Bedrock formats.
- Vertex AI
gemini-embedding-2and later models routed through:embedContent. - Final transformations that run after provider format conversion.
Guardrails for tool calls and OpenAI inline moderation
Regex prompt guards can opt in to scanning system prompts, messages, tool inputs, and tool outputs independently. This allows sensitive data and policy violations inside tool arguments or results to be detected without indiscriminately scanning every part of a request.
OpenAI providers can also inject OpenAI inline moderation into Chat Completions and Responses requests. Gateway configuration takes precedence over client-supplied moderation settings so clients cannot weaken the policy. For more information, see the Kubernetes and Standalone guardrail docs.
Dynamic forwarding, TCP, and policy backend URLs
Dynamic backends now support TCP traffic, CONNECT tunneling through another proxy, and an optional CEL target expression. Forward proxy authentication supports Proxy-Authorization and returns the expected 407 Proxy Authentication Required response on authentication failures. Backend connect timeout policy is now implemented end to end in both the proxy and controller.
Kubernetes policy callouts can use an inline URL instead of requiring an intermediate Service or AgentgatewayBackend. This applies to remote JWKS, tracing, OTLP access logs, external authorization, external processing, remote rate limiting, and backend tunnel proxies. HTTPS URLs automatically receive the corresponding inline backend TLS configuration.
Network-level external authorization is also available for connection-oriented traffic. For more information, see dynamic forward proxy and the Kubernetes API reference.
SPIFFE identities and backend authentication
Agentgateway can source its mTLS identity and trust bundle directly from the SPIFFE Workload API. The same rotating SVID can terminate frontend mTLS and authenticate outbound backend connections, while source.spiffeId exposes the authenticated identity to CEL policies. Startup fails closed if SPIFFE is enabled but an identity cannot be obtained.
Backend authentication can now mint a fresh short-lived JWT for each request, supporting key-pair JWT upstreams such as the Snowflake SQL API. JWT passthrough can optionally preserve the original token, backend authentication secrets reload when their files change, and AgentgatewayPolicy backend TLS can reference CA bundles in Secrets as well as ConfigMaps.
Cross App Access and OAuth token exchange can now configure scopes independently for each exchange leg, accept a configurable subject token type, and omit requested_token_type when an authorization server requires the RFC 8693 default behavior.
Other security and identity improvements include:
- Configurable sensitive request headers are redacted from trace and debug output, including headers created by CEL transformations.
- The controller Helm chart can optionally reduce cluster-wide permissions to read-only for most workload resources and scope write access to configured gateway namespaces. Existing cluster-wide write access remains the default.
Routing, rate limiting, and policy enhancements
This release expands routing and policy controls:
- Session affinity: A CEL expression can derive a session key from request data and consistently route requests with the same key to the same endpoint. Selection is deterministic across gateway replicas and does not require shared session state.
- Rate limiting: Kubernetes local rate limiting can apply multiple limits to a request. Remote rate limit descriptors can use a CEL-based
limitOverrideto calculate a different limit for each request. - Dynamic backend selection: A CEL expression can select the destination of a dynamic backend from request attributes or external-processing metadata, avoiding the need to rewrite the request authority.
- Backend connection timeouts: Backend connect timeouts can now be configured per backend.
Observability and operations
Distributed traces now include outbound spans for the upstream request and each policy callout, including external authorization, external processing, rate limiting, guardrails, and token exchange.
Metrics endpoints support Prometheus protobuf negotiation, and an opt-in native histogram mode is available. Standalone database access logging can explicitly store either LLM metadata or full prompt and completion content. LLM access logs include token timing and the matched prompt-guard pattern. agctl proxy profile can collect CPU and heap pprof profiles from a proxy.
The UI includes a rebuilt logs view, clearer multi-turn conversation rendering, tool call and result details, trajectory visualization, and a read-only storage mode that disables UI and API writes. Standalone can also reload file-backed backend authentication secrets without a restart.
🪲 Notable fixes
- Fixed dynamic listener port swaps without requiring a restart.
- Fixed AgentgatewayModel authorization on failover routes and InferencePool policy resolution for AI provider backends.
- Fixed Bedrock virtual models using the untransformed model in the upstream path.
- Fixed Bedrock streaming indexes,
top_ktranslation, invalid function inputs, image URL handling, and preservation of guardrail request structure. - Fixed Gemini usage extraction from Cloud Code response envelopes and preserved parallel tool calls across Gemini and Chat Completions conversions.
- Fixed Anthropic streaming input usage, first-delta roles, thinking budget bounds, and server-tool error handling.
- Fixed OpenAI file content conversion on the Vertex Gemini path and empty tool arguments in multi-turn requests.
- Fixed MCP discovery failures being hidden by
failOpen, authorization issuer metadata rewriting, upstream trace context, and routing of client responses to server-initiated requests. - Fixed A2A path and interface URL rewrites and added response telemetry for A2A v1.0 nested payloads.
- Fixed local API-key policy updates when file-mode keys are omitted.
- Fixed policy status when a target reference names a missing
sectionName. - Fixed rate-limited responses so they consistently include
x-ratelimitheaders. - Fixed upstream connection duration precision and retained full LLM proxy error phase information.
- Fixed the UI analytics summary request loop and CEL playground routing when the UI is exposed through a gateway.
Contributors
Thank you to everyone who contributed code, reviews, documentation, bug reports, and CI improvements for this release!
See the full contributor list below.
What's Changed
- auth: add jwtSign backend auth for keypair-JWT upstreams by @yordis in #2515
- feat(copilot): route Grok models through the Responses API by @iandvt in #2766
- chore: add max length to ttl policy field by @markuskobler in #2782
- llm: return the original golden tests by @howardjohn in #2783
- listener/tls: add authority key ID to dynamic CA certificates by @shashankram in #2786
- bump rmcp to 3.1.0 by @filintod in #2788
- llm: parse requests in golden tests by @howardjohn in #2787
- bedrock: Consistently image URLs across input types by @howardjohn in #2791
- vertex gemini: proper xhigh budget by @howardjohn in #2792
- Add AgentgatewayModel status reconciliation by @danehans in #2794
- Fix invalid header modifications by @howardjohn in #2775
- bedrock: handle invalid function inputs by @howardjohn in #2771
- config: drop MODEL_CATALOG_PATHS env var by @howardjohn in #2772
- docs: fix prompt guard webhook target examples by @nnennandukwe in #2796
- llm: make responses instructions golden actually use instructions by @howardjohn in #2795
- Stop bind if it transitions to internal bind by @howardjohn in #2735
- llm: preserve truncated tool calls by @howardjohn in #2776
- auth: fix examples for explicit OAuth subject tokens by @markuskobler in #2777
- api: remove deadcode type by @howardjohn in #2800
- llm: capture tool calls for all provider paths by @howardjohn in #2793
- llm: fix invalid max_token req case by @howardjohn in #2802
- llm: retain parallel tool calls for gemini<-->completions by @howardjohn in #2801
- llm: support cohere v4 embedding on bedrock by @howardjohn in #2820
- feat(loadbalancer): CEL-based stateless session persistence by @fengxsong in #2779
- build(deps-dev): bump fast-uri from 3.1.4 to 3.1.5 in /ui in the npm_and_yarn group across 1 directory by @dependabot[bot] in #2824
- llm: move more tests to goldens; recommend agents do the same by @howardjohn in #2822
- merge ai policy from AI backends with attached policies by @stevenctl in #2821
- Add support for listenerset postrouting policies by @howardjohn in #2739
- config: respect XDG_CONFIG_HOME by @howardjohn in #2823
- Fix initial model catalog refresh by @howardjohn in #2830
- llm: don't 503 the whole request when an Anthropic server tool is pre… by @sj-louw in #2804
- fix(llm): preserve buffered responses metadata by @iandvt in #2828
- Test
Programmeddefaults for rejected Gateways by @johananl in #2527 - chore:Fix incorrect code comment by @keithmattix in #2829
- feat(cel): allow error types to be serialized to strings without potentially-private details by @charles-dyfis-net in #2836
- llm: misc responses conversion correctness fixes by @howardjohn in #2842
- gemini: avoid filler text on tool call turns by @howardjohn in #2832
- perf(cel): avoid unnecessary materialize for Dynamic objects during has() by @charles-dyfis-net in #2835
- Add pod labels to standalone Helm chart by @howardjohn in #2844
- Register all request policies for CEL by @howardjohn in #2845
- chore(api): convert PolicyConditionType/Reason to string type aliases by @dongjiang1989 in #2815
- examples: add tier-aware semantic routing by @danehans in #2789
- auth: consolidate duplicated jwtSign / oauth token exchange code by @markuskobler in #2763
- proxy: make route policy application more consistent by @howardjohn in #2846
- crypto: add crypto module by @jgreeer in #2737
- llm: append [DONE] on chat -> anthropic flow by @howardjohn in #2843
- build(deps): bump cryptography from 49.0.0 to 50.0.0 in /examples/traffic-a2a/strands-agents in the uv group across 1 directory by @dependabot[bot] in #2847
- Import LiteLLM centralized credentials by @danehans in #2797
- llm: refactor guardrails by @howardjohn in #2848
- ui: fix placeholder example by @howardjohn in #2852
- fix(config): fail startup when a static local bind cannot be opened by @piyushbag in #2588
- llm: retain better HTTP error code for errors by @howardjohn in #2850
- 🤖 Fix :authority mutation no-op for CONNECT requests by @keithmattix in #2851
- fix: send x-ratelimit headers on all rate limited responses by @jgreeer in #2661
- Add h2c inference routing tests by @danehans in #2315
- jwt sign: misc followups from review by @howardjohn in #2849
- llm: support OpenAI inline moderation by @GuruduGanesh in #2519
- feat(cli): add agctl pprof profile command by @abhinavgautam01 in #2110
- feat(cel): Allow function call signature access for static checking by @charles-dyfis-net in #2855
- llm: skip encoding model id by @howardjohn in #2857
- cel: add LLM cost class helper by @mukeshbaphna in #1975
- Move off deprecated fetch_update by @howardjohn in #2859
- llm: route count tokens endpoint by default by @howardjohn in #2858
- llm: cap anthropic thinking budget by max tokens by @howardjohn in #2856
- controller: properly handle conflicts on internal GW/ListenerSet by @howardjohn in #2699
- Initial fuzzing by @howardjohn in #1853
- bedrock: correct top_k translation by @howardjohn in #2860
- fix(llm): extract Gemini usage from Cloud Code's
responseenvelope by @liorbenita2 in #2813 - build(deps): bump the cargo group across 1 directory with 3 updates by @dependabot[bot] in #2862
- AgentgatewayModel: properly configure authorization for failover by @howardjohn in #2861
- feat: use system theme as default by @guenhter in #2683
- Followups to session affinity by @howardjohn in #2825
- controller: expose remote rate-limit limitOverride by @FenjuFu in #2839
- support inline URL policy backends by @howardjohn in #2125
- examples: fix mcp-multiplex time target failing to start by @kristin-kronstain-brown in #2873
- controller: CEL validate that ports are numbers by @howardjohn in #2872
- Attempt to make copilot review less annoying by @howardjohn in #2869
- e2e: fix slow frontendtls test by @howardjohn in #2874
- e2e: add verbose logging of Istio libraries by @howardjohn in #2875
- ci: speed up e2e test builds by @howardjohn in #2878
- ci: include source in key by @howardjohn in #2882
- Split conformance job by @howardjohn in #2884
- bug: bedrock virtual models bypass transformed model for upstream path by @vascoccorreia in #2885
- ci: fully cache go by @howardjohn in #2893
- ci: pull mtime script remotely instead of vendoring by @howardjohn in #2895
- examples: add Redis semantic cache by @danehans in #2883
- api: remove redundant CEL uniqueness check on MCPBackend.targets by @GregCKrause in #2831
- ci: attempt windows build optimizations by @howardjohn in #2899
- deps: move off gcp git dep by @howardjohn in #2903
- e2e: avoid repeated create+delete Gateways by @howardjohn in #2897
- build(deps): bump js-yaml from 4.3.0 to 4.3.1 in /ui in the npm_and_yarn group across 1 directory by @dependabot[bot] in #2911
- Bump go dependencies by @howardjohn in #2900
- ci: attempt full rust cache for windows builds too by @howardjohn in #2898
- proxy: avoid blocking on otel drop by @howardjohn in #2876
- ci: run e2e ui tests by @howardjohn in #2901
- llm: expose normalized input tokens count in CEL by @howardjohn in #2880
- Add AgentgatewayModel into config dump by @howardjohn in #2907
- Revert "ci: run e2e ui tests" by @howardjohn in #2913
- readd conditional validation for ext services by @nfuden in #2896
- mcp: bump backend target
MaxItems32 -> 128 by @GregCKrause in #2916 - feat: import LiteLLM wildcard models by @FenjuFu in #2915
- bump cel-go for scanners by @nfuden in #2902
- fix: allow requested_token_type optional for Token Exchange (RFC 8693) by @sushisnow in #2892
- feat(controller): support Secret CA refs in AgentgatewayPolicy by @jvlxz in #2868
- ci: use full cache setup for all Go jobs by @howardjohn in #2917
- Log LLM token timing in access logs by @howardjohn in #2920
- standalone: more explicit LLM payload logging by @howardjohn in #2925
- ci: run ui e2e again by @howardjohn in #2923
- ci: invalidate stale Rust workspace cache artifacts by @howardjohn in #2928
- Fix flake in ui e2e by @howardjohn in #2929
- backend: support a CEL target expression on dynamic backends by @keithmattix in #2908
- llm: also parse custom tool calls by @howardjohn in #2930
- ui: better logs view by @howardjohn in #2888
- fix(ui): support macOS hybrid save override by @iandvt in #2938
- feat(transformations): post provider conversion transformation policies by @yanir75 in #2912
- deps(ui): pin Node to 24.17.0 across CI and Docker by @iandvt in #2940
- crypto: add symcrypt provider by @jgreeer in #2909
- chore(ui): replace prettier with biome by @iandvt in #2935
- llm: support Amazon Nova multimodal embeddings on Bedrock by @sankks11 in #2934
- proxy: drop special Istio TLV by @howardjohn in #2951
- build(ui): migrate from npm to pnpm by @iandvt in #2941
- ci: use fake keycloak for validation by @howardjohn in #2953
- [docs] update comments for mask streaming prompt guard by @artberger in #2954
- local rate limit: support multiple limits in k8s by @howardjohn in #2952
- fix: record upstream connect duration at full precision by @krisztianfekete in #2964
- feat: add shared Duration type to simplify CRD rules by @markuskobler in #2950
- build(ui): make UI builds consistent across workflows by @iandvt in #2961
- deps(ui): refresh pinned dependencies by @iandvt in #2960
- Add revisionHistoryLimit and dnsConfig to the controller Helm charts by @DrFaust92 in #2972
- sec: CVE bump go 1.26.5 by @iandvt in #2974
- cel: port parser bug fix by @howardjohn in #2970
- Scope ModelRouter to HTTPRoute by @keithmattix in #2973
- fix(mcp): route client JSON-RPC responses for server-initiated requests by @piyushbag in #2207
- fix(llm): honor final Anthropic stream input usage by @fengxsong in #2977
- fix(ui): forward LLM playground API key to MCP requests by @harukz in #2986
- fix(proxy): resolve InferenceRouting policy for AI provider_backend t… by @gaogaoSpark in #2976
- auth: configure Cross App Access subject token type by @markuskobler in #2750
- ui: clarify Client Setup description by @danehans in #2990
- agctl trace: handle oversized copy payloads by @howardjohn in #2997
- fix(status): surface policy targetRef sectionName missing on targets by @stevenctl in #2998
- llm: fix index on bedrock streaming by @howardjohn in #2971
- build: mark generated Go and schema files as generated by @markuskobler in #2999
- fix(a2a): correct interface URL rewriting when path rewrite policy is active by @dongjiang1989 in #2983
- Add ReadOnly storage mode to lock down UI/API writes by @jbohanon in #2988
- llm: set role on the first Anthropic streaming delta by @mautini in #3005
- fix: nightly builds by @markuskobler in #3010
- report errors on invalid inline jwks by @nfuden in #2991
- Bump to go 1.26.6 by @howardjohn in #3011
- serde: better errors for invalid JWT by @howardjohn in #3007
- controller: lock down valid JWKS targets a bit by @howardjohn in #3008
- policies: add network (connection) ext authz by @howardjohn in #3016
- llm: fix "Input should be a valid dictionary" when previous turn returned empty args by @mautini in #3002
- fix(llm): classify proxy errors by phase by @shubhamojha1 in #3015
- mcp: do not mask discovery failures on failOpen mode by @howardjohn in #3009
- jwt: require aud and iss claims by @howardjohn in #2993
- style(ui): expand Biome formatting and enable linting by @iandvt in #2957
- Feat protobuf metrics by @SajalBharadwaj in #3027
- bedrock: guardrail mutates in place preserving original structure by @stevenctl in #3021
- proxy: allow TCP traffic to use dynamic backend by @howardjohn in #3013
- docs: add egress proxy documentation by @howardjohn in #3014
- controller: write InferencePool status for pools referenced via AgentgatewayModel by @Geun-Oh in #3034
- fix: reconcile listener port swaps dynamically by @howardjohn in #3017
- feat(llm): native Gemini inbound API (generateContent, streamGenerateContent, countTokens) by @hjsauce in #2963
- feat(mcp): opt-in DNS rebinding protection for localhost Host/Origin by @piyushbag in #3022
- refactor(llm): expose stream termination to converters by @iandvt in #2994
- feat(xds): add custom request headers by @shubhamojha1 in #3052
- fix(mcp): derive upstream trace context from the gateway's active span by @stevenctl in #3059
- fix(llm): advertise the Responses format for the DeepSeek preset by @rajatvig in #3058
- fix: default file-mode apiKey policy keys to empty array on upsert by @Altamimi-Dev in #3048
- mesh: retain identity with nested TLS in HBONE by @howardjohn in #3069
- test(e2e): make test hooks and resources configurable by @markuskobler in #3070
- llm: defer re-encoding body by @howardjohn in #3071
- move catalog to not just be costs by @nfuden in #2927
- Add Anthropic Messages to OpenAI Responses conversion by @haowu1234 in #2689
- fix(a2a): populate response telemetry for A2A v1.0 nested payloads and record contextId by @Jpatel86 in #3073
- cel: port 310 limit recovery by @howardjohn in #3067
- feat: configure sensitive request headers by @shubhamojha1 in #3078
- auth: configure Cross App Access redemption scopes by @markuskobler in #2770
- fix(azure): support system-assigned managed identity by @iandvt in #3044
- deps: go 1.27 by @howardjohn in #3086
- fix(mcp): rewrite authorization server issuer metadata by @markuskobler in #3089
- Fix a2a path rewriting by @keithmattix in #3087
- fix(azure/managed-identity): reject multiple identity selectors by @iandvt in #3088
- release: drop unused caches by @howardjohn in #3043
- promptGuard: opt-in tool call coverage with scope by @stevenctl in #3000
- build(deps): bump oras.land/oras-go/v2 from 2.6.1 to 2.6.2 in /tools in the go_modules group across 1 directory by @dependabot[bot] in #3090
- ui: dep bump by @markuskobler in #3096
- tracing: add outbound span for all policy calls by @howardjohn in #3068
- metrics: add mode to enable native histograms by @howardjohn in #3079
- feat(llm): log which promptGuard pattern matched by @rajatvig in #3097
- dtrace: scope all outbound calls, not just ext authz by @howardjohn in #3100
- install: limit helm roles, allow namespaced write roles by @howardjohn in #3102
- [docs] update standalone Helm values by @artberger in #3104
- fix(util) modify convert_duration to clamp negative values to 0 by @jmcguire98 in #3103
- refactor: use simpler body logic for e2e test by @howardjohn in #3105
- dtrace: do not exit TUI if we get truncated by @howardjohn in #3099
- deploy: emit status on failures to write deployed objects by @howardjohn in #3101
- controller: implement backend TCP by @howardjohn in #3118
- delegation: require reference grant for Route<-->Route by @howardjohn in #3110
- llm: allow limiting authorized models on an API key by @howardjohn in #3107
- feat(auth): support forward proxy authentication via the
Proxy_Authorizationheader by @ericdbishop in #3098 - fix: cache InferencePool status to suppress redundant writes by @henrichter-sap in #3116
- Bump rust to 1.98 by @nfuden in #3122
- feat: source mTLS identity from the SPIFFE Workload API by @gcosgrave in #2640
- Bump golangci lint by @markuskobler in #3120
- Support CONNECT tunneling through dynamic proxy backends by @EItanya in #3095
- Full implement backend connect timeout policy by @howardjohn in #3124
- ui: fix cel playground when exposed over gateway by @howardjohn in #3138
- fix(llm): convert OpenAI file content parts on the Vertex Gemini path by @rajatvig in #3123
- feat(api): expose custom provider override by @shubhamojha1 in #3128
- Deeper Agent Substrate Integration by @keithmattix in #3051
- feat: reload backendAuth secrets when the file changes by @cbascom in #3132
- fix(ui): stop analytics summary request loop by @howardjohn in #3140
- api keys: reserve agentgateway.dev/ metadata prefix by @howardjohn in #3139
- schema: auto-generate metrics info by @howardjohn in #3141
- inference routing: do not trust client header by @howardjohn in #3144
- security: refine security policy a bit by @howardjohn in #3072
- test: cover multiple controller installations by @danehans in #2826
- examples: add Intune client verification and compliance by @danehans in #3012
- Keep substrate in sync by @keithmattix in #3145
- ui: better conversation view by @howardjohn in #3148
- ui: add new trajectory component by @howardjohn in #3149
- jwt: implement preserveToken by @howardjohn in #3106
- llm: route gemini-embedding-2 models to Vertex embedContent by @mautini in #2968
- docs: ban using examples for self-promotional content by @howardjohn in #3151
- dtace: correlate original body snapshot time by @howardjohn in #3150
- llm: translate openai cache markers to anthropic/bedrock by @howardjohn in #3109
- Add API-key scoped budgets for LLM traffic by @howardjohn in #3143
- cleanup controller Makefile by @markuskobler in #3171
- xds: reject non-FIPS RSA key parameters when running in FIPS mode by @ymesika in #3183
- preserve latest cache control when guardrails rewrites messages by @stevenctl in #3147
- chore: update golangci-lint plugins and enable unconvert linter by @dongjiang1989 in #3160
- mcp: handle recursive schemas in openapi translation by @howardjohn in #3154
- fix(tls/tcp over hbone): preserve hbone identity by @zac-nixon in #3156
- feat(helm): support controller topology spread constraints by @HasonoCell in #3159
- fix(pii): keep phone matches within one line by @jstar0 in #3162
- feat: support following proxy debug traces by @fengxsong in #3134
- metrics: minimal fix to return prometheus compat by @howardjohn in #3186
- feat(llm): add detect-only mode to Bedrock Guardrails by @theagenticguy in #2349
- Improve metrics endpoint handling by @markuskobler in #3185
- controller: allow cross-namespace InferencePool backendRefs by @koba1t in #3169
- fix postgres request log schema startup locking by @howardjohn in #2931
- hbone: improve error handling by @howardjohn in #3153
- guardrails: expose guardrails mask/rejection to cel access log by @stevenctl in #3189
- fix(ui): flag conflicting runtime listeners by @Math1987 in #2914
- fix split and nondeterministic frontend accessLog selection by @ymesika in #3199
- models: avoid naming collisions in keys by @howardjohn in #3205
- mcp: rate limit as JSON-RPC error by @stevenctl in #3146
- Add maxConcurrentRequests and maxConnections by @howardjohn in #3201
- Substrate Refinements by @keithmattix in #3174
New Contributors
- @nnennandukwe made their first contribution in #2796
- @charles-dyfis-net made their first contribution in #2836
- @mukeshbaphna made their first contribution in #1975
- @liorbenita2 made their first contribution in #2813
- @guenhter made their first contribution in #2683
- @FenjuFu made their first contribution in #2839
- @vascoccorreia made their first contribution in #2885
- @GregCKrause made their first contribution in #2831
- @sushisnow made their first contribution in #2892
- @jvlxz made their first contribution in #2868
- @sankks11 made their first contribution in #2934
- @DrFaust92 made their first contribution in #2972
- @harukz made their first contribution in #2986
- @gaogaoSpark made their first contribution in #2976
- @shubhamojha1 made their first contribution in #3015
- @SajalBharadwaj made their first contribution in #3027
- @Geun-Oh made their first contribution in #3034
- @hjsauce made their first contribution in #2963
- @Altamimi-Dev made their first contribution in #3048
- @Jpatel86 made their first contribution in #3073
- @ericdbishop made their first contribution in #3098
- @henrichter-sap made their first contribution in #3116
- @gcosgrave made their first contribution in #2640
- @cbascom made their first contribution in #3132
- @zac-nixon made their first contribution in #3156
- @HasonoCell made their first contribution in #3159
- @theagenticguy made their first contribution in #2349
- @koba1t made their first contribution in #3169
- @Math1987 made their first contribution in #2914
Full Changelog: v1.4.1...v1.5.0
🎉 Welcome to the 1.4.1 release of the agentgateway project!
Artifacts
Docker images are available:
cr.agentgateway.dev/agentgateway:v1.4.1cr.agentgateway.dev/controller:v1.4.1
Helm charts are available:
cr.agentgateway.dev/charts/agentgateway:v1.4.1cr.agentgateway.dev/charts/agentgateway-crds:v1.4.1cr.agentgateway.dev/charts/agentgateway-standalone:v1.4.1
Binaries for agentgateway and the agctl CLI are available below.
Quick Start
Follow the Kubernetes or Standalone quick start guide to get started!
Agentgateway v1.4.1
This release fixes a few bugs around LLM and MCP, increasing compatibility with the new 07-28-2026 specification.
Upgrade notes
- Top-level standalone
llmandmcplisteners may no longer share a port. To expose both on one port, attach them to the same configured gateway. - When a model references a named custom provider, that provider’s configured name is now used for custom model-catalog lookup. Catalog entries keyed under the generic
customprovider name may need to be updated. - OAuth token-exchange configurations that relied on an implicit fallback to validated JWT claims must explicitly select
jwt.rawToken.unredacted()as the subject-token source.
MCP improvements
- Added MCP Tasks support, including task retrieval, updates, cancellation, multiplexed task IDs, capability advertisement, RBAC enforcement, and task context in CEL and telemetry.
- MCP upstream responses are now validated against the expected response type.
- Asynchronous MCP errors are included in access logs.
- Fail-open fan-out requests now return an error when every upstream fails instead of producing an empty success.
- OpenAPI-derived MCP tools use the operation summary as their human-readable title.
- Synthetic internal session identifiers are no longer logged as MCP session IDs.
LLM and model handling
- Model catalogs now hot reload consistently when changed through either configuration files or the UI/configuration store.
- Fixed Amazon Bedrock requests containing tool-call history but no current tool definitions. Agentgateway now supplies the tool configuration Bedrock requires to continue these conversations.
- Fixed GitHub Copilot embedding responses that omit required OpenAI-compatible
objectandmodelfields. - Prompt-cache writes are no longer charged as both cache writes and ordinary input. Catalogs without a cache-write price fall back to the standard input rate.
Prompt guards
- Prompt guards now preserve structured Anthropic tool-use and tool-result content when masking text, preventing malformed requests and upstream 400 responses.
- Consecutive text blocks are evaluated together, allowing patterns spanning block boundaries to be detected.
- Non-text blocks and metadata such as Anthropic cache controls are preserved during masking.
- Zero-width regular-expression matches no longer insert masking placeholders throughout the input.
- Webhook prompt guards can access the full
llmRequestCEL variable and add CEL-computed headers through standalone configuration, Kubernetes CRDs, and xDS.
Authentication and security
- OAuth token exchange now fails closed when its configured subject-token source is missing or empty. Validated JWT claims are no longer used as an implicit fallback; configurations that intentionally use the validated token can select it explicitly with CEL.
- Cross-namespace JWKS backend references now honor the configured Gateway API
ReferenceGrantpolicy.
Configuration and operations
- Configuration storage and request logging can use separate databases when their settings differ, while identical configurations continue to share resources.
- Environment expansion now happens before configuration parsing, allowing variables in fields needed during startup.
- External authorization and processing services can return large response bodies without an artificial size limit.
Networking
- Added a full connection-handshake timeout so requests do not accumulate behind hung HTTP/2 connections.
What's Changed
- krt: fix Equals signatures by @stevenctl in #2704
- openapi: set title based on summary by @stevenctl in #2697
- refactor(controller): use apiequality.Semantic.DeepEqual for K8s API comparisons by @dongjiang1989 in #2687
- better cpu profile defaults by @ilrudie in #2682
- standalone: reject shared port for llm and mcp by @howardjohn in #2698
- fix: rewatch replaced model catalog files by @iploooox in #2702
- add Goose to Client Setup by @mkorbi in #2722
- Adding Configuration for GatewayClass and Controller Name by @jaiveerk in #2694
- Bump wiremock-rs by @markuskobler in #2726
- database: make connection pool size configurable by @sj-louw in #2705
- ext*: do not limit serve response size by @howardjohn in #2728
- database: split or share based on whether config is the same by @howardjohn in #2730
- client: avoid waiting on hung h2 connections by @howardjohn in #2729
- connect: scope internal bind lookup based on IP by @howardjohn in #2731
- llm: price prompt cache writes once by @danehans in #2727
- waypoint: accept traffic when serving as a weighted waypoint by @ymesika in #2716
- mcp: move off rmcp fork by @howardjohn in #2708
- webhook guardrail: expose llmRequest CEL var by @howardjohn in #2733
- fix(copilot): add OpenAI metadata to embedding responses by @iandvt in #2684
- feat: expose webhook prompt guard headers through Kubernetes CRD and xDS API (fixes #2718) by @ibariens in #2723
- Bump to rmcp v3 beta5 by @howardjohn in #2738
- fix(config): run shellexpend before parsing the config by @yanir75 in #2736
- jwks: add ref grant support by @howardjohn in #2706
- Add initial extensible standalone configuration importer by @danehans in #2605
- mcp tasks support by @stevenctl in #2745
- mcp: avoid logging synthetic session id by @howardjohn in #2743
- llm: remove guardrail deadcode by @howardjohn in #2746
- promptGuard: scan anthropic tool output by @stevenctl in #2732
- fix(bedrock): set toolConfig when messages carry tool history by @howardjohn in #2747
- llm: when using custom provider ref, override provider name by @howardjohn in #2754
- catalog: always hot reload by @howardjohn in #2753
- auth: fail closed when OAuth subject token source is missing by @markuskobler in #2740
- mcp: validate response body shape and log async errors by @howardjohn in #2744
- promptGuard: apply guard to merged consecutive text blocks by @stevenctl in #2755
New Contributors
- @ilrudie made their first contribution in #2682
- @mkorbi made their first contribution in #2722
- @jaiveerk made their first contribution in #2694
- @sj-louw made their first contribution in #2705
- @iandvt made their first contribution in #2684
- @ibariens made their first contribution in #2723
Full Changelog: v1.4.0...v1.4.1
🎉 Welcome to the 1.4.0 release of the agentgateway project!
This release brings a number of highlight features.
First, we have full support for the new MCP 2026-07-28 protocol, which we have been working with the MCP community to develop over the past year.
Additionally, Enterprise-Managed Authorization (also known as Cross App Access) for MCP has been added, enabling centralized access control for MCP servers.
Along the same authentication line, a flexible and feature-rich OAuth token exchange backend authentication mode has been added, enabling a number of security authentication strategies.
The standalone mode has a number of enhancements: configuration created in the UI can now be stored in the database (sqlite for local usage, postgres for remote or HA setups), avoiding the need for a persistent disk. A new gateways configuration replaces the low-level binds API and enables serving LLM, MCP, generic routes, and the UI on the same port; additionally, this enables authentication (such as OIDC) for the UI. A new agentgateway-standalone Helm chart is available for deploying standalone agentgateway in Kubernetes.
The Kubernetes deployment has a new (off by default) experimental AgentgatewayModel API, bringing the standalone LLM experience to Kubernetes. AgentgatewayModel allows serving a number of LLM models under the same gateway, routed based on the model name in the request. This offers a higher level API than AgentgatewayBackend, simplifying the common configuration cases; the existing API remains available.
The release also includes Gateway API v1.6 support, fault injection, richer guardrails, and many other enhancements and bug fixes.
Artifacts
Docker images are available:
cr.agentgateway.dev/agentgateway:v1.4.0cr.agentgateway.dev/controller:v1.4.0
Helm charts are available:
cr.agentgateway.dev/charts/agentgateway:v1.4.0cr.agentgateway.dev/charts/agentgateway-crds:v1.4.0cr.agentgateway.dev/charts/agentgateway-standalone:v1.4.0
Binaries are available below.
Quick Start
Follow the Kubernetes or Standalone quick start guide to get started.
🔥 Breaking changes
Gateway API v1.6 and TCPRoute v1
Agentgateway now builds against Gateway API v1.6, and the Kubernetes controller uses the v1 version of TCPRoute instead of v1alpha2. Re-apply the Gateway API CRDs that match this release before you upgrade.
MCP request-phase guardrail rejections return HTTP 200
When an MCP guardrail rejects a request during the request phase, agentgateway now returns the rejection as an HTTP 200 with a JSON-RPC error body, matching the existing response-phase behavior. Update any clients or tests that expected a non-200 status for request-phase rejections.
For more information, see Kubernetes MCP guardrails and Standalone MCP guardrails.
(Standalone) auth.location no longer nests expression
The auth.location configuration no longer uses a double-nested expression field. Update any policies that set a custom token location to use the flattened form.
musl container images removed
The musl-based container image variants are no longer published. Switch to the standard (glibc) images.
The binary releases are still built with musl.
🔒 Security
Stateful MCP sessions can cross routes and overwrite the authorization policy
This release addresses the security advisory GHSA-mvgg-jvj2-4frq of a High (8.1) severity impacting MCP authorization.
Please see the advisory for details and mitigation steps.
Thank you to Aonan Guan for this report.
Request/Response Body CEL policies
A report was made to us that authorization policies written against the request.body or response.body CEL variables may result in unexpected behavior due to the truncation behavior of these attributes.
This is a security recommendation on the usage of these fields in authorization policies, and not a bug in agentgateway.
This behavior has also changed in this release; see below:
| Attribute | Version | Behavior |
|---|---|---|
request.body |
1.3.x and earlier | Truncated to http.maxBufferSize (default 2MB) |
request.body |
1.4 | Not available if truncated |
request.truncatedBody |
1.3.x and earlier | Not available |
request.truncatedBody |
1.4 | Truncated to http.maxBufferSize (default 2MB) |
The split between body and truncatedBody allows more control over the body when it exceeds the maximum buffer size.
In all cases, usage of these fields in authorization requests is prone to mistakes. For example, consider this request:
string(request.truncatedBody).contains("attacker-payload")
If the request is over 2MB, the attacker-payload may be present but missed.
Additionally, request encoding schemes (compression, etc) can result in additional bypasses.
When using these fields in authorization policies, ensure the policy correctly degrades in these cases.
Thank you to Aonan Guan for this report.
🌟 New features
MCP protocol 2026-07-28 support
Agentgateway adds support for the upcoming MCP 2026-07-28 protocol version.
- Stateful and stateless servers: Agentgateway supports both stateful and stateless MCP servers, including closing the SEP-2575 server-stateless conformance gap and skipping the synthetic
initializehandshake for modern requests. - Trace context propagation: Distributed trace context propagates through the MCP
_metafield. - MCP Apps: Basic support for MCP Apps, including multiplexing fixes for app-originated tool calls.
- Capability and multiplexing improvements: Preserved multi-resource tool result capabilities for modern clients, multi-target subscriptions and listen, and opaque resource URI multiplexing.
Because MCP 2026-07-28 support is new in this release, most of it is not yet covered by a dedicated guide. For the fields available today, see the Kubernetes API reference and Standalone configuration reference.
Cross App Access for MCP
Agentgateway supports MCP Enterprise-Managed Authorization through the OAuth Identity Assertion Authorization Grant, also known as Cross App Access or ID-JAG. An enterprise identity provider can broker access between a client application and the MCP server without the end user completing a separate OAuth flow for each downstream app.
For more information, see Kubernetes Cross App Access and Standalone Cross App Access.
OAuth token exchange backend authentication
Agentgateway can exchange an incoming token for a backend credential by using RFC 8693 OAuth 2.0 token exchange and the RFC 7523 JWT bearer grant. This release adds Kubernetes controller support, custom token types and OAuth 2.1 exchange defaults, the ability to inject multiple secret-sourced headers, and an override for the resolved secret key.
For more information, see Kubernetes OAuth token exchange and Standalone OAuth token exchange.
New MCP authentication providers
Agentgateway adds native MCP authentication provider support for Microsoft Entra ID. Standalone also adds native providers for Descope and authentik.
The entra provider bridges OAuth behaviors that Microsoft Entra ID implements differently from the MCP authorization specification, including serving RFC 8414 metadata from Entra's OIDC discovery document, stripping the RFC 8707 resource parameter, and short-circuiting Dynamic Client Registration with your pre-registered application ID.
For more information, see Kubernetes MCP authentication, Standalone MCP authentication, and the Standalone Descope integration guide.
New standalone gateways configuration model
Standalone configuration gains a new top-level gateways concept that unifies the UI, LLM, MCP, and routes so that they can share a single listener and port. gateways supersedes binds; existing binds configuration continues to work, and the UI offers a one-click migration from binds to gateways.
This release also adds simpler host and TLS configuration, the ability to serve LLM and MCP traffic on the same port, and an internal bind mode with wildcard fallback. For the full configuration surface, see the Standalone configuration reference.
Virtual keys from ConfigMaps and hashed keys
Virtual keys can now be sourced from a ConfigMap in addition to a Secret, and API keys can be stored as SHA-256 hashes so that raw key material never needs to live in the cluster or in plaintext configuration.
For more information, see Kubernetes virtual keys and Standalone virtual keys.
CEL enhancements
This release expands the CEL surface across policy, telemetry, and transformations:
- Custom CEL functions: Register custom CEL functions for use in policies.
- CEL filters for telemetry: An opt-in CEL filter selects which requests emit OpenTelemetry spans, and CEL filters decouple OTLP log fields and filtering from stdout logging.
- New CEL context and functions: Access inbound
CONNECTrequest headers throughsource.connectHeaders, and use a CEL replace mode for header transformations.
For the full CEL surface, see the Kubernetes CEL reference and Standalone CEL reference.
Fault injection: request delay
A new delay traffic policy injects latency before a request is forwarded to the backend, for fault-injection and chaos testing. The delay duration accepts a duration string or a CEL expression that returns a duration, or a number interpreted as milliseconds. Injected delay counts against the request timeout.
For more information, see Kubernetes fault injection and Standalone fault injection.
AWS assume-role session tags and session name
AWS assumeRole backend authentication supports STS session tags and a configurable RoleSessionName. Both the session name (sessionNameExpression) and per-tag values (tags[].expression) can be set from CEL expressions evaluated per request, so you can propagate identity attributes such as jwt.sub into the assumed AWS session.
For more information, see Standalone backend authentication. For Kubernetes fields, see the Kubernetes API reference.
Guardrail enhancements
Guardrails include several new operational and policy controls:
- Backend connection policy for callouts: A
BackendConnectionPolicycontrols the TCP, TLS, HTTP, and tunnel settings agentgateway uses when it calls out to a guardrail service, and is available on the OpenAI moderation, Bedrock guardrails, and Google Model Armor policies. - Default callout timeouts: Guardrail callouts now apply a default timeout.
- Improved logs and UI: Guardrail decisions surface more clearly in logs and the UI.
failureModefor external processing: External processing supports fail-open or fail-closed behavior.
For more information, see Kubernetes LLM guardrails, Standalone prompt guards, Kubernetes MCP guardrails, and Standalone MCP guardrails.
External processing enhancements
The Kubernetes controller supports metadataContext, requestAttributes, and responseAttributes for external processing, and extProc exposes a failureMode for fail-open or fail-closed behavior.
For more information, see Kubernetes external processing and Standalone external processing.
LLM gateway enhancements
- Frontend TLS with multiple CAs: Client certificate validation can trust multiple CAs.
- Bedrock: Added Responses-to-Bedrock image translation, sanitized tool names that exceed the 64-character Converse limit, and propagated cache-write tokens to the access log.
- Gemini: Fixed embeddings handling and
generateContentmodel and usage extraction in detect mode. - A2A v1.0: Support for the A2A v1.0 agent card format in URL rewriting.
- Azure AI Foundry: Support for Anthropic endpoints on Foundry.
For the list of supported providers, see Kubernetes LLM providers and Standalone LLM providers.
Deployment, operations, and packaging
- Standalone Helm chart: A new
agentgateway-standaloneHelm chart deploys standalone agentgateway, with support for a metrics service andServiceMonitor. For more information, see Standalone Helm deployment. - DaemonSet workloads: Kubernetes can deploy the data plane as a
DaemonSet. - Sidecars in control plane pods: A Helm
extraContainersvalue runs sidecar containers in control plane pods. - Metrics scraping: Kubernetes can scrape proxy metrics with a
PodMonitor, and a newagentgateway_controller_build_infometric reports controller build details. - Database logging: Standalone logging writes both successes and errors to the SQL database, with optimized write performance.
For more information, see Kubernetes Helm installation, Kubernetes control plane metrics, and Standalone metrics.
Contributors
Thank you to everyone who contributed code, reviews, documentation, bug reports, and CI improvements for this release!
See the full contributor list below.
What's Changed
- costs: add prometheus counter metric by @howardjohn in #2243
- ext authz: cache HTTP as well as gRPC by @howardjohn in #2283
- ui: fix pricing catalog by @howardjohn in #2241
- dtrace: log body for streaming llm too by @howardjohn in #2287
- cel: expose CONNECT request headers as source.connectHeaders by @shashankram in #2285
- feat(telemetry): add opt-in CEL filter for OTel spans by @inercia in #2295
- fix(foundry): add support for anthropic endpoints on foundry by @Darkflame72 in #2190
- Centralize LLM provider format capabilities by @danehans in #2288
- ext authz: add custom body for http payloads by @howardjohn in #2284
- ui: fix virtual API key copy button by @yalindogusahin in #2294
- tracing: expose filter in Kubernetes by @howardjohn in #2300
- dtrace: add copy details button by @howardjohn in #2289
- controller: fix MCP Guardrails not referencing AGBE by @howardjohn in #2307
- build(deps): bump golang.org/x/image from 0.28.0 to 0.38.0 in the go_modules group across 1 directory by @dependabot[bot] in #2303
- admin: allow directly routing to internal admin interface by @howardjohn in #2299
- llm: fix detect and audio endpoints by @howardjohn in #2301
- Add standalone agentgateway Helm chart by @kevin-shelaga in #2255
- agctl: trim 100mb off the binary by @howardjohn in #2290
- proxy: terminate outer TLS before serving CONNECT tunnels by @shashankram in #2308
- api key: allow sha256 encoded keys by @howardjohn in #2240
- Custom CEL functions by @howardjohn in #2230
- test cost configmaps by @nfuden in #2306
- agctl: move trace and config command logic under proxy subcommand by @GuruduGanesh in #2197
- fix(llm): apply pathPrefix for passthrough routes on custom-host backends by @onematchfox in #2151
- Support A2A v1.0 agent card format in URL rewriting by @timonschmidt in #2251
- security: fix timing attacks in auth and add admin IP allowlist by @rafael31p in #2191
- proxy: avoid noisy TLS logs by @howardjohn in #2311
- auth: add RFC 8693 token exchange backend auth by @gilad9366 in #2189
- Release standalone Helm chart by @kevin-shelaga in #2320
- feat(ui): detect Claude subscription misuse and show warning by @rajv2er in #2321
- Allow conformance GatewayClass overrides by @danehans in #2329
- guardrails: avoid unnecessary regex guard allocations by @stevenctl in #2328
- Drop and test large stack sizes by @howardjohn in #2336
- binds: add internal bind mode and wildcard fallback by @shashankram in #2326
- pprof: use frame-pointers by @howardjohn in #2332
- build(deps): bump the cargo-weekly group with 11 updates by @dependabot[bot] in #2352
- cel: make duration(duration) and identity function by @howardjohn in #2354
- llm: remove unnecessary async/.await from process_streaming by @markuskobler in #2356
- [feat] GWAPI FrontendTLS: Support multiple CAs for the client cert validation in agentgateway by @krinkinmu in #2173
- dtrace: include frontend policy selection by @howardjohn in #2355
- mcp: accept get stream when only a subset of servers have 405 by @howardjohn in #2342
- buffer: copy buffer limit around more by @howardjohn in #2353
- mcp: add modern MCP release foundation by @filintod in #2345
- Move incoming_connect_tunnel_reenters_internal_bind test to ephemeral port by @howardjohn in #2362
- fix extproc attributes and grpc initial metadata by @jenshu in #2364
- bump to rust 1.96.1 by @nfuden in #2390
- feat(buffer): Allow streaming on bodies larger than the limit by @yanir75 in #2358
- controller: support ext_proc metadataContext/requestAttributes/responseAttributes by @fengxsong in #2369
- chore: Bump
kube-api-linterversion and fixgoseclinter by @dongjiang1989 in #2366 - fix: honor TLS for tracing HTTPS endpoints by @jstar0 in #2372
- llm: register transformation CEL expression by @howardjohn in #2393
- policy: support targeting frontend policies to a Gateway port by @shashankram in #2363
- guardrails: default timeouts for callouts by @nfuden in #2388
- Initial support for MCP 2026-07-28 by @howardjohn in #2365
- standalone: use simple schema macro more by @howardjohn in #2392
- otlp logs: decouple fields and filter from stdout by @howardjohn in #2313
- gateway: support internal bind mode via annotation by @shashankram in #2380
- controller: move to TCPRoute v1 by @howardjohn in #2389
- deployer: populate InternalPorts in GatewayIRFrom fallback by @shashankram in #2396
- standalone: add dependency tracking for config by @howardjohn in #2327
- tools: bump dependencies by @howardjohn in #2394
- docs: Update tool names in Authorization example by @aakashsbhatia2 in #2344
- refactor: replace sort.Slice with slices.Sort for natural ordering by @dongjiang1989 in #2367
- Use k8s:enum more by @howardjohn in #2405
- config: better error messages by @howardjohn in #2406
- add azureauth types to k8s side by @jgreeer in #2337
- Return request-phase MCP guardrail rejections as 200, matching the response phase by @christensenjairus in #2331
- llm: refactor standalone model router by @howardjohn in #2330
- chore(lint): upgrade golangci-lint and its plugins; run clippy before formatting by @markuskobler in #2381
- openapi: support application/octet-stream request bodies in MCP tool generation by @dhruvbehl in #2400
- Feat: Add Descope as an MCP authentication provider by @joshwertheim in #2376
- fix: add default timeout for oauthTokenExchange token endpoint by @jgreeer in #2410
- Bedrock: sanitize tool names exceeding 64-char Converse limit by @devmittal02 in #2384
- auth.location: remove double nested
expressionby @howardjohn in #2411 - dtrace: show final status and error by @howardjohn in #2412
- Expand OAuth token exchange backend auth by @markuskobler in #2338
- Bump istio version to fix broken startup due to TCPRoute 1.6.0 by @keithmattix in #2432
- build(deps): bump the cargo-weekly group with 5 updates by @dependabot[bot] in #2438
- feat(metrics): add agentgateway_controller_build_info metric and fix SetRegistry lifecycle bug by @dongjiang1989 in #2399
- feat: add support for AWS session tags & RoleSessionName by @Lyqed in #2435
- build(deps): bump cmov from 0.5.3 to 0.5.4 by @dependabot[bot] in #2439
- Code of conduct: clarify anti-issue agent stance by @howardjohn in #2441
- standalone: support LLM and MCP on the same port by @howardjohn in #2409
- controller: drop deadcode from opencensus by @howardjohn in #2440
- Refactor LLM translation paths by @howardjohn in #2415
- auth: OAuth Identity Assertion Authorization Grant Support (ID-JAG / "Cross App Access" / XAA) by @markuskobler in #2436
- fix(mcp): skip upstream JSON-RPC error frames in FailOpen fanout merge by @syf2211 in #2428
- Refactor examples folder by @howardjohn in #2242
- Resolve Istio CRDs via
go mod downloadby @johananl in #2446 - Bump to head of gateway-api release-1.6 by @howardjohn in #2360
- fix(bedrock): propagate cache-write tokens to access log for completi… by @ankkod in #2444
- standalone: allow simpler host + TLS configuration by @howardjohn in #2450
- test: add goleak to detect goroutine leaks in controller packages by @dongjiang1989 in #2419
- fix(jwt): no owners from policy fetch for targetSelectors by @jgreeer in #2452
- build(deps): bump golang.org/x/image from 0.38.0 to 0.41.0 in the go_modules group across 1 directory by @dependabot[bot] in #2454
- gemini: do not fail on embeddings by @howardjohn in #2455
- added demos and examples for cross app access for keycloak, xaa.dev, … by @christian-posta in #2449
- build: build embedded UI automatically and add UI=0 opt-out by @jbohanon in #2457
- added examples for oauth token exchange and jwt bearer by @christian-posta in #2424
- fix: Remove duplicate CEL validation markers by @dongjiang1989 in #2461
- llm: split credential cache from providers by @howardjohn in #2459
- Move end to end test flows to integration tests by @howardjohn in #2468
- fix: apply discoveryNamespaceSelectors filter to AgentgatewayBackend and AgentgatewayPolicy informers by @julien-aubert in #2442
- fix(llm): decode Content-Encoding on request bodies before JSON parse by @westover in #2404
- Split LLM into its own crate by @howardjohn in #2469
- llm: move auth back to main crate by @howardjohn in #2470
- feat: support CEL expressions for AWS session tag values by @Lyqed in #2447
- ui: update to tsc 7 by @howardjohn in #2471
- standalone: log errors to sql db as well, not just success by @howardjohn in #2476
- controller: OAuth Token Exchange support by @markuskobler in #2458
- Hardcode test CA certificate by @johananl in #2481
- Allow simple host+policy config in logs and traces too by @howardjohn in #2474
- Rust 1.97 by @howardjohn in #2479
- standalone: Add new "Gateways" concept, unifying UI + traffic on one port by @howardjohn in #2473
- fix: responses api with date based azure api + get_path_for_model rewrite by @jgreeer in #2493
- ui: misc improvements by @howardjohn in #2499
- Bump conformance to fix test flakes by @howardjohn in #2500
- controller: better logging by @howardjohn in #2491
- xds: version resources by @howardjohn in #2497
- Enable the generated crd description len to be configured by @markuskobler in #2489
- mcp: skip synthetic initialize for modern requests by @stevenctl in #2477
- azure: log construction errors when all credentials fail by @jgreeer in #2502
- Add LLM semantic routing example by @danehans in #2486
- bump go oras 2.6.2 by @nfuden in #2517
- Document Rust toolchain requirement by @johananl in #2516
- examples: suppress semantic router domain defaults by @danehans in #2506
- build(deps): bump the cargo-weekly group with 6 updates by @dependabot[bot] in #2514
- examples: use buffered semantic routing ExtProc by @danehans in #2521
- Expose OAuth Token Exchange builder by @markuskobler in #2522
- auth: inline OAuth Token Exchange & Cross App Access endpoint config by @markuskobler in #2529
- build(deps): bump soupsieve from 2.8.3 to 2.8.4 in /examples/traffic-a2a/strands-agents in the uv group across 1 directory by @dependabot[bot] in #2485
- mcp: basic support for Apps by @stevenctl in #2475
- fix flake in debug trace test by @stevenctl in #2523
- Wait for base gateway before port forwarding by @johananl in #2496
- build(deps): bump click, pillow, pygments in examples/traffic-a2a by @markuskobler in #2540
- feat(helm): add extraContainers value for running sidecars in control plane pods by @zackwy in #2542
- examples: support Responses semantic routing by @danehans in #2532
- Added descriptions by @kristin-kronstain-brown in #2545
- mcp: add "prefix: never" to fix multiplexing for app originated tool calls by @stevenctl in #2531
- .github: nightly agctl artifacts by @nfuden in #2556
- fix(standalone): classify dual-role policies for backend targets by @piyushbag in #2543
- Add support for Responses→Bedrock images translation by @npolshakova in #2548
- examples: tune semantic routing escalation by @danehans in #2557
- feat: dynamic (CEL) RoleSessionName for AWS assume-role auth by @Lyqed in #2508
- Fix reasoning in messages -> completions by @howardjohn in #2504
- schema: fix gen-docs to support map types by @howardjohn in #2544
- Match repeated HTTP headers individually by @howardjohn in #2535
- standalone: optimize logging writes to database by @howardjohn in #2484
- docs: link client guide from routing example by @danehans in #2549
- close SEP-2575 server-stateless conformance gap by @filintod in #2417
- Add DaemonSet workload support by @johananl in #2208
- feat(auth): allow overriding the resolved secret key for oauth and gcp credential refs by @yordis in #2518
- build(deps): bump mcp from 1.26.0 to 1.28.1 in /examples/traffic-a2a/strands-agents in the uv group across 1 directory by @dependabot[bot] in #2561
- fix(llm): rewrite Vertex rawPredict for any publisher by @piyushbag in #2539
- controller: update the cost catalog configmap loading by @nfuden in #2530
- mcp: preserve MRTR capabilities for modern clients by @stevenctl in #2559
- Expose
failureModefor extProc by @aakashsbhatia2 in #2551 - feat(mcp): add authentik as an MCP authentication provider by @aubinmora in #2567
- mcp: propagate trace context via _meta (SEP-414) by @stevenctl in #2520
- avoid dropping target-level policies for mcp by @stevenctl in #2414
- release: drop musl images by @howardjohn in #2574
- Add dtrace event for trace sampling decision by @howardjohn in #2573
- fix(llm): extract Gemini generateContent model and usage in detect mode by @emeresHub in #2571
- Improve guardrails logs and UI by @howardjohn in #2575
- deps: bump quick-xml & quinn-proto by @markuskobler in #2578
- Add agctl nightly by @npolshakova in #2577
- Support bundled certs by @howardjohn in #2579
- add configmaps as a source for virtualkeys by @nfuden in #2570
- fix(mcp): multiplex opaque resource URIs by @piyushbag in #2538
- Fix quick-xml vulnerabilities by @danehans in #2584
- mcp: tighten check for passthrough routes by @howardjohn in #2576
- mcp: multi-target subscriptions/listen by @stevenctl in #2599
- deps: bump weekly cargo for jul 20 2026 by @nfuden in #2597
- crdgen: shorten generated CRD descriptions by @markuskobler in #2598
- Fixed links by @kristin-kronstain-brown in #2602
- security: document test-only rmcp advisory by @danehans in #2585
- feat(oauth): support custom token types and OAuth 2.1 exchange defaults by @markuskobler in #2580
- chore: Add Gateway-TCP conformance profile by @davidjumani in #2604
- S390x support by @satyamg1620 in #2149
- build(deps): bump js-yaml from 4.2.0 to 4.3.0 in /ui in the npm_and_yarn group across 1 directory by @dependabot[bot] in #2611
- controller: add Cross App Access auth support by @markuskobler in #2534
- cel: do not expose body when exceeds buffer by @howardjohn in #2615
- aws auth: per-backend SigV4 signing region by @stevenctl in #2610
- add delay policy for fault injection by @stevenctl in #2613
- helm: scrape proxy metrics with PodMonitor by @mmmommm in #2591
- feat(transformation): add CEL replace mode for headers by @piyushbag in #2589
- standalone: support writing config to database by @howardjohn in #2603
- build(deps): bump the cargo-weekly group across 1 directory with 4 updates by @dependabot[bot] in #2612
- proxy: avoid double poll on Body by @howardjohn in #2582
- auth: make crossAppAccess subject token source configurable by @sladkani in #2558
- controller: Narrow prompt guard auxiliary backend auth by @markuskobler in #2614
- build(deps): bump pyasn1 from 0.6.3 to 0.6.4 in /examples/traffic-a2a/strands-agents in the uv group across 1 directory by @dependabot[bot] in #2632
- build(deps-dev): bump fast-uri from 3.1.2 to 3.1.4 in /ui in the npm_and_yarn group across 1 directory by @dependabot[bot] in #2631
- Decompress request body on model router path by @howardjohn in #2635
- feat(mcp): add Microsoft Entra ID as a native MCP authentication provider by @asaliyog in #2408
- controller: expose cross-app subject token source by @markuskobler in #2629
- feat(llm): configurable webhook guardrail headers and path via CEL by @robin-fiddler in #2595
- llm: allow logging user AND prompt by @howardjohn in #2638
- auth: add default cloud auth fetch timeout by @howardjohn in #2637
- feat(ui): add passthrough backend auth policy editor by @jbohanon in #2601
- standalone: do not default expose admin interface by @howardjohn in #2642
- ui: remove redundant pnpm lock by @howardjohn in #2653
- ui: remove diff on delete button by @howardjohn in #2652
- RIP StarHistory by @howardjohn in #2654
- fix(mcp): emit explicit SSE message events by @haowu1234 in #2644
- ui: align status on logs by @howardjohn in #2639
- hybrid mode: allow writing policies to the database by @howardjohn in #2651
- Fallback to custom UUID generation by @howardjohn in #2657
- Add support for notify for multi-replica postgres by @howardjohn in #2656
- feat(monitoring): add support for metrics service and ServiceMonitor configuration in agentgateway-standalone helm chart by @yanir75 in #2647
- test: fix flake in dtrace tests by @howardjohn in #2660
- add cli commands to support config migrations by @nfuden in #2617
- Update readme diagram by @howardjohn in #2655
- feat(backendAuth): support injecting multiple secret-sourced headers by @yordis in #2316
- chore: updates to README by @jbohanon in #2666
- feat(a2a): add response telemetry by @haowu1234 in #2646
- ui: fix the port mode for mcp in standalone ui by @jbohanon in #2669
- New
AgentgatewayModelAPI by @keithmattix in #2583 - standalone: more hybrid mode revamps by @howardjohn in #2670
- feat(llm): use Gemini native generateContent/streamGenerateContent endpoints on Vertex by @htimur in #2023
- llm: make the error parsing permissive for completions by @howardjohn in #2673
- standalone: revamp helm chart for v1.4 by @howardjohn in #2671
- add tool calls in telemetry by @lcopi in #2233
- perf(jwks): parallelize JWKS fetches with bounded concurrency by @dongjiang1989 in #2594
- build(deps): bump postcss from 8.5.15 to 8.5.23 in /ui in the npm_and_yarn group across 1 directory by @dependabot[bot] in #2675
- docs: recommend against Deny policies by @howardjohn in #2677
- Bump deps ahead of v1.4 by @howardjohn in #2674
- cel: if the body exceeds the buffer size, expose it as a separate attribute by @howardjohn in #2678
- standalone: rename configStore to storage by @howardjohn in #2679
- chore: add max length to certain policy fields by @keithmattix in #2680
- oauth: support certificate credentials for private_key_jwt by @markuskobler in #2620
- mcp: pin session to backend by @howardjohn in #2691
- messages -> completions: allow reasoning for more models by @howardjohn in #2692
- ci: ensure all crates are included in default workspace by @howardjohn in #2693
- llm: allow non-gpt models to use reasoning with tools+messages by @howardjohn in #2695
New Contributors
- @Darkflame72 made their first contribution in #2190
- @yalindogusahin made their first contribution in #2294
- @GuruduGanesh made their first contribution in #2197
- @timonschmidt made their first contribution in #2251
- @rafael31p made their first contribution in #2191
- @rajv2er made their first contribution in #2321
- @krinkinmu made their first contribution in #2173
- @dongjiang1989 made their first contribution in #2366
- @jstar0 made their first contribution in #2372
- @aakashsbhatia2 made their first contribution in #2344
- @christensenjairus made their first contribution in #2331
- @dhruvbehl made their first contribution in #2400
- @joshwertheim made their first contribution in #2376
- @devmittal02 made their first contribution in #2384
- @Lyqed made their first contribution in #2435
- @syf2211 made their first contribution in #2428
- @julien-aubert made their first contribution in #2442
- @westover made their first contribution in #2404
- @zackwy made their first contribution in #2542
- @piyushbag made their first contribution in #2543
- @yordis made their first contribution in #2518
- @aubinmora made their first contribution in #2567
- @emeresHub made their first contribution in #2571
- @davidjumani made their first contribution in #2604
- @satyamg1620 made their first contribution in #2149
- @mmmommm made their first contribution in #2591
- @sladkani made their first contribution in #2558
- @asaliyog made their first contribution in #2408
- @robin-fiddler made their first contribution in #2595
- @haowu1234 made their first contribution in #2644
- @htimur made their first contribution in #2023
- @lcopi made their first contribution in #2233
Full Changelog: v1.3.1...v1.4.0
@ Every self-hosted release, weekly
One Monday email with the week's stable releases across the directory.
More in API Gateway
AgentSafeSelf-hosted gateway intercepting AI agent HTTP actions, asking Decionis for authorization, forwarding only approved requests with signed evidence.
ai-apiSelf-hosted AI gateway unifying Azure OpenAI, OpenAI, Anthropic, and Gemini behind one OpenAI-compatible API with revocable credentials and cost tracking.
Ailin¹ Collective IntelligenceSelf-hosted open-source engine coordinating 100,000+ AI models through 31 strategies with verifiable consensus and OpenAI-compatible API.
AISIX OSSSelf-hosted, open-source AI gateway with one OpenAI-compatible API for 100+ LLM providers, routing, guardrails, caching, rate limits, and observability — built in Rust.
anythingmcpSelf-hosted gateway turning APIs and databases into MCP tools for AI assistants, with hundreds of ready connectors and access controls.
api200Self-hosted API gateway simplifying third-party integrations with auth, caching, retries, mocks, schema watching, and MCP support via one URL.
apimeSelf-hosted, open-source WhatsApp REST API gateway in Go with multi-instance dashboard, webhooks, and Node.js SDK.
apisixOpen-source, self-hosted API and AI gateway built on NGINX and etcd, with hot-loaded plugins, dynamic routing, load balancing and rate limiting.
bifrostOpen-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.
BricksLLMOpen-source Go AI gateway that meters LLM traffic with per-key rate limits, spend caps, and PII masking, covering OpenAI-style APIs, Anthropic, Azure and vLLM.
CatGPT-GatewaySelf-hosted gateway turning ChatGPT/Claude browser sessions into an OpenAI-compatible API with tool calling, vision, and file attachments.
ccLoadOpen-source gateway you host that combines AI keys and subscriptions into one endpoint with failover, protocol translation, and usage costs for coding assistants.


0 comments