Multi-Tenant Agent Architecture

Tier: Advanced · Hands-on time: ~35 minutes

This area now has first-party canon: AWS's 2026 guidance on building multi-tenant agents with Bedrock AgentCore documents the pooled-runtime-plus-centralized-MCP-gateway pattern explicitly, which means the pattern conversation can start from published architecture rather than folklore. The design problem is classic SaaS multi-tenancy with new failure modes stacked on top — an agent that holds memory, calls tools, and consumes expensive tokens can leak, starve, or overspend across tenant boundaries in ways a stateless API never could.

When to use

• A SaaS product is adding agentic features for all customers and the runtime-sharing decision must be made before build.
• Per-tenant contracts differsome customers demand dedicated isolation while the long tail expects pooled pricing.
• A pooled deployment shows noisy-neighbor symptomsone tenant's usage degrading others' latency or blowing shared quotas.
• Don't reach for this when the platform serves internal teams of one org with uniform trust — standard environment separation covers it; revisit when external tenants arrive.

Prerequisites

• Tenant facts: count and growth, contractual isolation obligations per segment, usage skew expectations (the top tenant's share of load).
• The single-tenant agent designtopology (pb-35), memory (pb-36), tools (pb-37), approvals (pb-38) — because multi-tenancy is a dimension across all four.
• The AWS multi-tenant AgentCore pattern doc (or your platform's equivalent) open as the reference point.
• A chat assistant for isolation red-teaming.

The workflow

1. 1.

   Decide pooled, siloed, or tiered — per layer, not globally. Runtime, memory store, vector indexes, tool gateway, and model quota can each be pooled or siloed independently. The published pattern — pooled runtime with a centralized MCP gateway enforcing per-tenant scoping — is the cost-efficient default; siloed layers are bought where contracts or blast-radius demand them. A tiered offering (pooled standard, siloed premium/regulated) is often the honest commercial answer; design both paths from one codebase.

2. 2.

   Design tenant identity to travel end to end. Every request carries tenant context from the app boundary through the agent, into every tool call and every retrieval. The rule that survives audits: tools and indexes enforce tenancy from propagated identity — never from the agent's prompt remembering which tenant it is serving. Prompts are not a security boundary.

3. 3.

   Scope memory and retrieval per tenant. Session state, long-term memory, and vector indexes are keyed and filtered by tenant (pb-31's index-per-tenant versus metadata-filter fork applies verbatim to agent memory). Define tenant offboarding as a first-class flow: all memory, checkpoints, embeddings, and traces for the departing tenant enumerable and deletable.

4. 4.

   Design noisy-neighbor controls before the first incident. Model tokens are the scarce shared resource: per-tenant rate limits and token budgets at the gateway (virtual keys per tenant make this enforceable and measurable), queue fairness for long-running tasks, and quota headroom math that assumes your most enthusiastic tenant triples overnight.

5. 5.

   Make cost per tenant observable by design. Tag every model call, tool invocation, and storage object with tenant ID at creation; gateway virtual keys give per-tenant spend natively. This is the difference between "we can price agentic features per tenant" and a quarterly archaeology project — design-time attribution, per this track's FinOps principle, is drastically cheaper than retrofit.

6. 6.

   Red-team the isolation story. Anita, taking orbit-docs-copilot's agent features multi-tenant:

   Copy

   Attack this multi-tenant agent design (attached: pooled AgentCore-style
   runtime, centralized MCP gateway with per-tenant virtual keys,
   metadata-filtered shared vector index, tenant-keyed Postgres memory).
   
   Find cross-tenant paths in:
   1. Memory — any flow where tenant A's session or long-term memory can
      enter a prompt serving tenant B (include checkpoint resume and
      summarization jobs).
   2. Tools — any tool call where tenant scoping relies on prompt content
      rather than propagated identity.
   3. Retrieval — filter-bypass paths in the shared index, including
      cache layers that might serve cross-tenant cached results.
   4. Cost/quota — how tenant A exhausts a shared quota or budget and
      what tenant B experiences.
   5. Offboarding — what a departed tenant's data leaves behind in
      checkpoints, caches, traces, and embeddings.
   Severity, concrete scenario, and closing design change for each.
   

   Triage findings yourself; each becomes a design change or a documented, justified acceptance.

7. 7.

   Write the decision record. Per-layer pooled/siloed table with reasons, identity-propagation diagram, per-tenant quota and budget scheme, offboarding flow, and revisit triggers (tenant count thresholds, first regulated-tenant contract, noisy-neighbor incident). This ADR is what the commercial team's isolation promises must match.

Verify before you trust it

• Cross-tenant retrieval is tested hostile. Two synthetic tenants in the shared index; queries as tenant A engineered to surface tenant B's content must return nothing — including through any caching layer, which is the routinely-forgotten path.
• Identity propagation is proven at the tool. Capture a tool-side log for a test request and confirm the enforced tenant came from propagated identity, with a negative test showing a prompt-claimed tenant is ignored.
• The token budget trips. Drive one synthetic tenant past its budget and confirm throttling fires for them while a second tenant's latency stays flat — that pair of observations is the noisy-neighbor control working.
• Offboarding leaves nothing. Offboard a synthetic tenant, then sweep memory, checkpoints, index, caches, and traces for residue. Findings here before launch are cheap; after launch they are disclosures.
• Platform pattern claims verified in current docs. Runtime session semantics, gateway virtual-key behavior, and per-tenant limits come from the vendor's live documentation — this pattern literature is new and moving.

Restricted-environment variant

When the "tenants" are your consultancy's clients, the stakes sharpen: MSAs often prohibit co-mingling outright, forcing siloed layers per client regardless of cost logic — extract those clauses before designing, and let the tiered model absorb the difference. Client-tenancy endpoints (Bedrock or Azure OpenAI inside each client's subscription) naturally silo model traffic and spend per client. All isolation testing uses synthetic tenants only. For the strictest engagements, the entire agent stack — runtime, gateway, memory, index — deploys inside each client's boundary, and the pooled economics conversation ends there; a local Ollama-based rig remains the development environment either way.