AKS Architecture

This page describes the checked-in AKS deployment, not an attestation of a live cluster. For provisioning and deployment commands, see Deploy to AKS.

Deployed components

The application Gateway routes to Frontend, API, and MCP Services. Worker is a background control-plane workload, not another public Gateway backend. Its configured floor is two replicas; the HPA permits two to three using CPU 70% and memory 80%. API and Worker both read and write the same EF-backed PostgreSQL stores and use the shared workspace PVC. API/Worker use the privileged control-plane identity; AgentHost has a separate identity without Key Vault roles, and MCP mounts no secrets.

Preview browser traffic uses a separate preview Gateway, not the application Gateway or an API reverse proxy. The network diagram and credential diagram below show these boundaries.

AgentHost warm-pool lifecycle

Worker runs in pod-per-run: coordinator children execute in AgentHost pods rather than in-process on Worker. The shared agentweaver-agent-host warm pool keeps two pods pre-warmed (k8s/base/sandbox-warmpool-agenthost.yaml).

Warm pods boot without a RunId and enter standby. The executor creates a claim, persists its identity, and waits for pod binding. It probes /healthz for HTTP reachability: standby also returns 200. One-time POST /configure supplies run identity, bounded credentials, purpose, and the workspace contract. Successful configuration completes setup and marks IsReady before returning. A2A traffic is gated by that ready state; there is no second post-configuration health poll.

Before the first A2A request, dispatch is fenced to the run lifecycle generation, project, submitting user, agent, accepted provider snapshot, and an opaque dispatch identifier. Concurrent callers join the same launch. Readiness, configuration, or endpoint failure can replace the claim once while delivery is still known not to have started; after the streaming request begins, Agentweaver never replays the turn. A stale launch cannot satisfy a newer generation. Exhaustion records one retryable, credential-redacted agent_host_unavailable terminal outcome for that dispatch.

Shared execution uses /workspace. Assembly Build/Test uses LocalReadOnly: fetch an immutable source ref, verify base commit and tree, and check out detached into the disk-backed /local-workspace emptyDir. Its preview uses the same verified checkout. This mode forbids write-back, not local build output. Current Worker configuration also enables pod-local implementation work; it is not just a future seam. Assistant-purpose setup skips project checkout and ordinary agent setup.

The first valid /configure atomically binds the pod before setup finishes. Later valid attempts return 409, including after setup failure. Configuration cannot require the turn bearer it delivers; subsequent streamed turns require that per-run bearer. Network reachability and production mTLS are separate controls: the preview Gateway's allowed port range also includes 8088.

A successful Assistant turn retains its configured pod and renews MCP authorization separately for later turns. Failure/cancellation releases it. Ordinary release deletes the claim and revokes/unregisters run capabilities; an active preview can defer cleanup. Turn completion therefore does not always mean claim deletion.

The executor uses AgentHostWarmPoolRef (default agentweaver-agent-host), not per-run templates or warm pools. Grounding: KubernetesSandboxExecutor.cs claim/configuration and release paths, AgentHostReadinessProbe.cs, AgentHostStartupService.cs, and RemoteOperatorAssistantAgent.cs; detailed citations accompany the diagram review.

Networking flow

Inbound request path

Both TLS Gateways listen on 443 using approuting-istio. Each Gateway → HTTPRoute → Service → pod chain is explicit. API manages preview resources but is not a browser traffic hop. Exact paths and longer prefixes win over the frontend / catch-all.

Route	Matches	Service → pod
API	Prefix /api, /auth, /openapi; exact /.well-known/oauth-authorization-server, /.well-known/openid-configuration, /oauth/authorize, /oauth/token, /oauth/register, /oauth/resume, /oauth/revoke, /oauth/jwks	agentweaver-api:8080 → API:8080
MCP	Prefix /mcp; exact /.well-known/oauth-protected-resource and /.well-known/oauth-protected-resource/mcp	agentweaver-mcp:8080 → MCP:8080
MCP health	Exact /mcp/health, rewritten to /healthz	agentweaver-mcp:8080 → MCP:8080
Frontend	Prefix /	agentweaver-frontend:80 → Frontend:8080
Preview	Dynamic {token}-preview.{zone} hostname on the separate Gateway; upstream hostname rewritten to localhost	preview-{token}:80 → run-labeled AgentHost target port

There is no blanket /oauth prefix route. The same diagram covers the effective policy boundaries below; a second traffic diagram is unnecessary.

Security model

Network security — Cilium NetworkPolicy

The cluster uses Azure CNI Overlay and Cilium (--network-dataplane cilium). Application Routing uses an Istio-based gateway data plane; this does not mean workload pods have Istio sidecars or an ambient service mesh.

NetworkPolicy rules

Boundary	Selector / permitted source	Effect
Application ingress	Application Gateway pods or aks-istio-ingress namespace	API, MCP, Frontend pod TCP 8080
Application egress	API, MCP, Frontend	Deny by default, with explicit DNS/internal/external rules
AgentHost default	app=agentweaver-agent-host	Deny ingress except additive allows
A2A ingress	Same-namespace API or Worker pods	TCP 8088 to AgentHost
Preview ingress	Same-namespace pods labeled gateway.networking.k8s.io/gateway-name=agentweaver-preview-gateway	TCP 3000–9000 inclusive to AgentHost
AgentHost internal egress	Destination app=agentweaver-api or app=agentweaver-mcp	TCP 8080 using pod selectors
AgentHost DNS	kube-system + k8s-app=kube-dns, or 10.0.0.10/32	UDP/TCP 53
AgentHost public-address HTTPS	IPv4/IPv6 CIDR rules below	TCP 443, not a GitHub-only CIDR

Sandbox isolation

The selected policies are additive allows, not intersecting restrictions. The preview range includes 8088, so network policy does not make API/Worker the only sources able to reach the A2A listener. Production mTLS/client-certificate authentication is a separate control.

API ingress requires both app=agentweaver-agent-host and agentweaver.dev/sandbox=true; MCP ingress requires app.kubernetes.io/component=agent-host. The template carries these labels. Destination pod selectors permit API/MCP access without granting broad private network access. Under the documented Cilium path, CIDR/world permission does not replace cluster-pod identity permission.

Public-address HTTPS permits 0.0.0.0/0 except 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, and ::/0 except fc00::/7, fe80::/10. These are the actual exclusions, not every nonpublic range: 100.64.0.0/10 is not listed. The manifests document the Kata DNS-hook limitation and explicit IP/port fallback. A narrower-looking FQDN policy does not make this effective union FQDN-only. HTTPS reachability does not grant Key Vault authorization.

Agent execution uses Kata VM-isolated pods (runtimeClassName: kata-vm-isolation) claimed through SandboxClaim (extensions.agents.x-k8s.io/v1beta1). The in-cluster API selects KubernetesSandboxExecutor when KUBERNETES_SERVICE_HOST is present. See Sandbox verification.

Non-root containers

API and Frontend run as UID 1000 with runAsNonRoot: true and dropped capabilities. API also sets allowPrivilegeEscalation: false.

Secrets management

Azure Workload Identity lets API/Worker use Key Vault without embedded Azure credentials. The trusted control-plane broker redeems a run's purpose-bound capability snapshot and fences authority before and after retrieval. It supplies copilotCredential through one-time /configure, or supplies the selected BYOK configuration. Repository credentials, A2A turn tokens, and Assistant MCP tokens remain separate. AgentHost does not resolve an ambient user's token or read Key Vault.

API and Worker ServiceAccounts federate through AKS OIDC to agentweaver-api-identity, which has Key Vault Secrets User and Secrets Officer grants. Their Kubernetes RBAC remains distinct; Worker does not inherit API preview-management permissions. agentweaver-agent-host federates to agentweaver-agenthost-identity, with no Key Vault roles. Provisioning removes its legacy federation to the privileged identity. The broker is trusted API/Worker application code, not a separate public service.

One static SecretProviderClass, agentweaver-secrets, configures CSI app-secret delivery for API and Worker. It is not the vault itself. It mounts files and syncs a Kubernetes Secret. The manifest includes the internal API key, provider-key signing key, telemetry connection string, and Repo/Copilot App configuration.

Value	Consumption
mcp-api-key	Startup wrapper reads /mnt/secrets-store/mcp-api-key for internal API authentication; not a hosted MCP bearer key
ai-execution-provider-key-signing-key	Synchronized Kubernetes secretKeyRef supplies AiExecution__ProviderKeySigningKey
OAuth signing/encryption certificate families	API runtime SecretClient loads usable Key Vault versions with active/previous overlap; not CSI delivery

The CSI mount triggers synchronization; do not describe all values as file-only. MCP mounts no secrets and accepts only Agentweaver-minted broker JWTs for the exact /mcp audience and mcp:invoke scope. Worker does not host the API OAuth certificate loading path.

CSI app-secret rotation polling is two minutes. Create the provider-key signing key once with high-entropy material and rotate only with a coordinated API rollout: serving replicas must agree, and rotation invalidates prepared execution contexts. AgentHost receives no OAuth-client-secret mount. See Configuration for credential import, migration, and recovery procedures.

Authentication

Microsoft Entra ID provides browser identity; end users are not issued API keys.

1. The user selects Sign in with Microsoft Entra ID.
2. API redirects to the configured Entra application.
3. Entra returns to https://<host>/auth/entra/callback; API establishes platform identity and roles.
4. Repository discovery and GitHub project creation use a separate Repo App handoff and opaque selection code.
5. Model access follows the project versus personal provider hierarchy.

The renderer derives Auth__Entra__RedirectUri and Auth__Entra__FrontendUrl from the public host, without localhost fallback. Register the exact callback under the Entra app's publicClient platform; npm run azure:setup-entra-app -- --redirect-uri https://<host>/auth/entra/callback prepares registration. Do not change a production identity registration without its owner's approval.

MCP authentication

MCP forwards authorized caller context to API at http://agentweaver-api:8080. MCP OAuth, repository authorization, and model capabilities are separate boundaries. See MCP connection and Authentication.

External dependencies

Service	Purpose	Boundary
GitHub APIs	App capability and permitted repository operations	Control-plane HTTPS; AgentHost public-address HTTPS is not FQDN-only
GitHub browser origin	Repo/Copilot consent handoffs	Browser connectivity, not pod network authorization
Azure Key Vault	CSI and trusted runtime credentials/certificates	Reachability plus privileged workload identity; no AgentHost vault grant
Azure Container Registry	Image pulls	Kubelet/cluster ACR authorization, not pod credential delivery
PostgreSQL	Durable application state	TCP 5432; private subnet or rendered public-access FQDN policy according to deployment
Telemetry endpoint	Monitoring export	Deployment-specific telemetry configuration; see Operations

Storage model

PostgreSQL (primary data store)

API and Worker use EF-backed application stores via MemoryDbContext, including projects, runs, revisions, workflow state, memory, decisions, OAuth state, and durable events. This is neither an API-read/Worker-write split nor a production Dapper/EF partition. Provisioning stores connection configuration in agentweaver-postgres, not in images.

Connection string key	Precedence	Used by
ConnectionStrings__Postgres	First	Shared EF stores
ConnectionStrings__MemoryDb	Fallback	Same EF stores
Database__ConnectionString	Final fallback	Same EF stores

PostgreSQL supports multiple API replicas with RollingUpdate, without SQLite's single-writer deployment constraint.

Workspace volume

The shared agentweaver-workspace PVC is 50 GiB RWX Azure Files, with StorageClass azurefile-csi-premium-uid1000, mounted at /workspace. AgentHost separately mounts an 8 GiB disk-backed execution-scratch emptyDir at /local-workspace; it is not durable shared state.

PVC: agentweaver-workspace (Azure Files, RWX)
  storageClass: azurefile-csi-premium-uid1000
  mountPath: /workspace
  |
  +-- .home/                (shared app/runtime state; no GitHub token mirror)
  +-- worktrees/            (git worktrees per run)
  +-- <project workspaces>  (project working directories)

EF Core migrations

API and Worker init containers (migrate-memory-db) run /app/efbundle --verbose -- --postgres-migrations before their main containers start. They use the API image and injected ConnectionStrings__MemoryDb / ConnectionStrings__Postgres from agentweaver-postgres. No connection string is embedded in the image or manifest. Local design-time commands default to SQLite; use --postgres-migrations only with configured PostgreSQL credentials.

For release-candidate acceptance, deploy the committed candidate with npm run azure:deploy-from-commit -- <candidate-sha> and run representative integration plus feature-specific API/UI tests against that exact-SHA deployment before preparing or publishing the release. See RELEASING.md.

Ephemeral storage for testing

For throwaway SQLite testing, set Database__Provider=Sqlite and replace the shared workspace volume with:

volumes:
  - name: workspace
    emptyDir: {}

Data is lost on pod restart. SQLite requires one replica and Recreate strategy to avoid write contention; this is not a production storage migration procedure.

Diagram details and constraints

Element	Contract
title	AKS network: two ingress planes
takeaway	Routes select Services; additive policies bound AgentHost access, not a vault credential path.
n-app-title	APPLICATION ORIGIN • EXACT MATCHES / LONGER PREFIXES BEAT /
n-preview-title	PREVIEW ORIGIN • API MANAGES RESOURCES, NOT BROWSER TRAFFIC
n-egress-title	EFFECTIVE AGENTHOST POLICY UNION
Client	Client
Client	Browser / MCP
Client	Public app origin
App Gateway	App Gateway
App Gateway	agentweaver-gateway
App Gateway	TLS :443
HTTPRoutes	HTTPRoutes
HTTPRoutes	API / MCP / /
HTTPRoutes	Exact OAuth routes below
Services	Services
Services	API/MCP :8080
Services	Frontend :80
App pods	App pods
App pods	API / MCP / web
App pods	All target :8080
Browser	Browser
Browser	{token}-preview
Browser	Separate hostname
Preview GW	Preview GW
Preview GW	preview-gateway
HTTPRoute	HTTPRoute
HTTPRoute	preview-{token}
HTTPRoute	Host rewrite: localhost
Service	Service
Service	preview-{token}:80
Service	Run-label selector
AgentHost	AgentHost
AgentHost	Preview target port
AgentHost	Gateway: 3000–9000
API / MCP	API / MCP
API / MCP	Selector-based TCP 8080
API / MCP	DNS: UDP/TCP 53 separately
Public HTTPS	Public HTTPS
Public HTTPS	TCP 443, IPv4 + IPv6
Public HTTPS	Private/link-local exclusions
Additive allows	Additive allows
Additive allows	API/Worker → A2A :8088
Additive allows	Preview range includes 8088
n-internal	TCP 8080
n-public	TCP 443
n-routes-heading	APP ROUTES
n-routes-body	API: /api, /auth, /openapi + exact OAuth/discovery. MCP: /mcp + resource metadata; /mcp/health → /healthz.
n-exclusions-heading	EGRESS EXCLUSIONS
n-exclusions-body	10/8, 172.16/12, 192.168/16, 169.254/16; fc00::/7, fe80::/10. Not FQDN-only; no vault authority implied.
notes	[object Object]; [object Object]
groups	[object Object]; [object Object]; [object Object]

Diagram details and constraints

Element	Contract
title	AgentHost: bind once, serve ready
takeaway	HTTP reachability is not setup readiness; claim lifetime can span Assistant turns.
l-launch-title	01 CONTROL-PLANE LAUNCH
l-setup-title	02 PURPOSE-BOUND SETUP
l-turns-title	03 TURN AND CLAIM LIFETIME
Claim warm pod	Claim warm pod
Claim warm pod	Persist claim; wait for binding
Claim warm pod	Shared pool: 2 warm pods
Probe listener	Probe listener
Probe listener	/healthz success: reachable
Probe listener	200 can mean standby
Configure once	Configure once
Configure once	Run, purpose and capabilities
Configure once	POST /configure
Select workspace	Select workspace
Select workspace	Shared or verified local checkout
Select workspace	LocalReadOnly: no write-back
Finish setup	Finish setup
Finish setup	Assistant skips project checkout
Finish setup	Accepted binding stays consumed
Ready for A2A	Ready for A2A
Ready for A2A	Setup finished before response
Ready for A2A	IsReady gates traffic
Stream a turn	Stream a turn
Stream a turn	Run-bound bearer authentication
Stream a turn	message:stream
Retain Assistant	Retain Assistant
Retain Assistant	Successful turn keeps its pod
Retain Assistant	Renew MCP token, not configure
Release claim	Release claim
Release claim	Unregister and revoke capabilities
Release claim	Active preview defers cleanup
l1	bound
l2	reachable
l3	accepted
l4	prepare
l5	complete
l6	dispatch
l7	success
l8	next turn
l9	run ends / failure
l-409-heading	CONFIGURATION BOUNDARY
l-409-body	Later valid configure attempts return 409, even after accepted setup fails.
notes	[object Object]
groups	[object Object]; [object Object]; [object Object]

Diagram details and constraints

Element	Contract
title	Credentials: authority stays in the control plane
takeaway	Separate Azure identities; separate Copilot, repository, turn and MCP capabilities.
s-control-title	TRUSTED CONTROL PLANE
s-host-title	ISOLATED AGENTHOST
s-delivery-title	DISTINCT DELIVERY CHANNELS
API + Worker SAs	API + Worker SAs
API + Worker SAs	Separate Kubernetes RBAC
API + Worker SAs	AKS OIDC federation
API managed identity	API managed identity
API managed identity	Secrets User + Secrets Officer
API managed identity	agentweaver-api-identity
Azure Key Vault	Azure Key Vault
Azure Key Vault	Credential and app-secret authority
Azure Key Vault	Runtime and CSI consumers
Capability broker	Capability broker
Capability broker	Fence run + purpose before/after read
Capability broker	No ambient user-token lookup
AgentHost identity	AgentHost identity
AgentHost identity	Separate ServiceAccount federation
AgentHost identity	No Key Vault role assignments
AgentHost runtime	AgentHost runtime
AgentHost runtime	One-time /configure delivery
AgentHost runtime	Copilot or BYOK; repo separate
CSI app secrets	CSI app secrets
CSI app secrets	SecretProviderClass configuration
CSI app secrets	Files + synced secretKeyRef
OAuth certificates	OAuth certificates
OAuth certificates	API runtime SecretClient
OAuth certificates	Usable active / previous versions
MCP resource server	MCP resource server
MCP resource server	Agentweaver broker JWT only
MCP resource server	Exact /mcp + mcp:invoke
s1	federate
s2	authorize
s3	credential
s4	configure
s5	pod identity
s6	app secrets
s7	cert versions
s8	Assistant JWT
s-separation-heading	NO AGENTHOST → VAULT EDGE
s-separation-body	Network reachability is not authorization. Repository, A2A and MCP tokens are not Copilot credentials.
notes	[object Object]
groups	[object Object]; [object Object]; [object Object]

Visual model

AKS deployment

Structured source · Editable draw.io