Productizing AI Agents When Every Tenant Needs Its Own Isolated Runtime

Shared containers can't safely hold agents, so isolate each tenant.

· 9 min read
Cover illustration for “Productizing AI Agents When Every Tenant Needs Its Own Isolated Runtime”
Agent Architecture · November 11, 2026 · 9 min read · 1,953 words

To deploy persistent AI agents per user at scale, each tenant needs an isolated runtime boundary, tenant-scoped memory and credentials, and state that lasts between sessions without keeping compute running. Productizing an agent means choosing that architecture before launch. Shared-container separation built only on application code is too weak for processes that execute code, hold secrets, and act on customers’ behalf.

Why agents break the assumptions that made shared-container SaaS safe

Traditional SaaS multi-tenancy mostly treats isolation as a database problem. Tenant A must not read tenant B’s rows, while the application controls the shared compute. Agents break that frame. They keep workspaces alive, execute generated code, use browsers, and call external services with the authority customers give them. A boundary failure can therefore expose both another tenant’s information and the tools used to act on it.

Zylos AI’s architecture research identifies four coupled risks, including stateful execution, tool authority, model inference as a side channel, and prompt or context contamination. A recycled pod can retain workspace files, cached secrets, or in-memory context. Shared model calls create another route for leakage when tenant context is not strictly scoped. An agent may then act on instructions from another tenant’s knowledge base. The problem resembles an operating system’s job more than a database’s, because separation must cover processes, memory, files, networks, and capabilities.

The four isolation layers that must hold simultaneously

Three out of four is not isolation. Compute, data and memory, tool credentials, and observability each have their own way to fail. The compute layer needs an isolated execution environment for each run, rather than application checks inside a shared container. Before an environment is used for another tenant, its filesystem state, shared memory, and environment variables also need to be cleared.

Data isolation extends beyond the main database. Each workspace, conversation history, vector memory, and structured state needs a boundary of its own. Vector namespaces need tenant identifiers on every vector and filters enforced on every read, while separate indices keep tenants further apart but cost more. A similarity search without the tenant filter can return another customer’s material.

Credentials and traces need equally explicit ownership. Tool credentials must be scoped to a single run rather than exposed as tenant-wide, long-lived tokens. Traces, token usage, and spending must remain attributable to the tenant so that debugging, billing, and audit records work. Axonius explicitly identified per-tenant cost tracking as a requirement when designing its agent offering on Bedrock AgentCore. An isolated filesystem cannot compensate for a credential or trace assigned to the wrong customer.

Silo, pool, and bridge: choosing an isolation pattern before writing any code

Diagram: Silo, Pool, Bridge: Three Isolation Patterns Compared. Visualizes: Show three mutually exclusive architectural isolation patterns for multi-tenant AI agents, ranked from strongest to most economical isolation.

Silo gives each tenant a dedicated runtime, process space, and lifecycle. It has strong noisy-neighbor protection and makes audit boundaries simpler, but costs more to build and run. Axonius chose this approach for hundreds of customer environments already separated into dedicated VPCs. Each agent uses only its customer’s data and integrates with the existing authentication and authorization module on that tenant’s EC2 instance, with JWT validation.

Pool trades dedicated infrastructure for logical separation. Scoped identifiers, access policies, and data partitioning lower overhead and let customers join without infrastructure provisioning. Application correctness becomes the weak link. A missing tenant filter or header can cross the boundary. Bedrock AgentCore’s healthcare pool example uses a Tier → Tenant → User hierarchy for its knowledge-base documents, memory, model access, and cost tracking.

Bridge separates selected components while sharing others. A dedicated runtime can, for example, use a common knowledge base, putting stronger protection where the risk justifies its cost. Zylos’s guidance distinguishes internal tooling from customer-facing SaaS, recommending microVM-level compute isolation, separate vector namespaces, and per-tenant credential vaults for SaaS, with separate data stores for regulated tenants.

What actually belongs inside an agent container versus what belongs outside it

Inside each agent’s runtime boundary, only its execution loop, identity bundle, and live session state need to differ. The loop calls the model and applies tools. Identity includes the system prompt, agent-scoped credentials, and configuration, while live state includes the conversation buffer, working memory, and scratch files. Everything else needs a reason to be there. Message routing, scheduling, databases, reverse proxies, and observability can stay outside and serve multiple agents without being duplicated per runtime.

Credential handling needs a further split between the agent worker and its sidecar. In the credential-proxy pattern, the worker has no outbound access except to a sidecar over loopback. The sidecar injects authentication, forwards requests, and strips authentication metadata from responses, keeping raw credentials out of the worker’s reach. Identity belongs to the isolated agent, but secrets need not belong inside its process.

AgentCore’s design passes identifiers, tiers, preferences, and entitlements through custom HTTP headers. Aligning the runtime boundary with the identity boundary makes attribution structural and reduces its dependence on application code.

How do you deploy persistent agents per user at scale with snapshot-restore?

Giving each session a microVM looks costly if every session has to keep running. Snapshot-restore changes that calculation. Firecracker can suspend a sandbox while preserving memory and filesystem state, then restore its files, installed packages, browser sessions, and in-memory context. When environments are suspended, they use storage without using active compute.

Managed platforms differ in what they preserve. DigitalOcean Managed Agents, in public preview in 2026, isolates generated code in microVMs, pauses idle sessions, and resumes in well under a second with files and state intact. AgentCore Runtime provides per-session microVMs with isolated filesystems, but persistent storage is an opt-in preview feature. Session isolation and cross-session persistence are separate buying criteria. A platform can provide one without providing the other by default.

Human review benefits from the same mechanism. An agent can pause mid-task for hours without spending that time on active compute, then resume with its working state preserved. Warm pools cut the wait for new work to start by keeping pre-initialized environments ready. Scheduling can be shared while each session remains isolated.

Sandboxing LLM-generated code: why parse-time filtering is not a substitute for execution-time isolation

Generated code is still arbitrary code when it comes from a helpful model. A prompt injection that causes malicious code to run does not need to defeat every safeguard in the product. It only needs an execution environment that grants too much access. System prompts cannot provide that environment’s security boundary. Model behavior and runtime containment solve different problems.

Pattern checks cannot replace containment either. Regex-based filters cannot work out exactly what every alias, dynamic import, indirect access pattern, or code path will do. More careful analysis may improve inspection, but inspection is not execution-time isolation. The sandbox must limit what code can reach when it runs, with default-deny outbound networking as well as compute separation. An isolated process with unrestricted network access still has an escape route for data.

Untrusted task execution also needs a disposable lifecycle. Its environment and disk should be destroyed when the task ends. This differs from preserving an agent’s trusted session state between interactions.

Computer-use agents on Windows, Linux, and Android: isolation requirements specific to desktop and phone automation

Filesystem and network separation are insufficient when an agent can read a desktop, inspect the accessibility tree, or inject input. Microsoft’s Execution Containers, announced at Build 2026 as an early preview, separate agent execution from the user’s desktop, clipboard, and input devices. Display and input isolation prevent one session from reading another’s screen state or acting through its controls.

The screen is also a data surface. An agent clicking through an EHR or payer portal may send screenshots containing protected health information, or PHI, to a model. The model endpoint, screenshot store, and logs must sit under a business associate agreement, or BAA, and the portal’s terms must permit screen automation. Moving from an API to a screenshot moves sensitive data through a different path with the same privacy obligation.

Evaluation now spans desktop and phone environments. OSWorld, WindowsAgentArena, and AndroidWorld cover computer-use tasks across those settings. Human takeover has its own safety needs because a person may need to use the desktop while keeping the agent’s state intact. That handoff depends on session design.

HIPAA, audit trails, and the compliance cost of getting isolation right in regulated industries

Healthcare makes architecture a compliance decision from the start. HHS says an AI vendor that handles PHI is a business associate under HIPAA and must meet the obligations of a business associate. That obligation is distinct from proposed changes to the Security Rule. As of late 2026, the update addressing agentic AI remained proposed, with final action targeted for July 2027. A proposed rule is not a completed compliance milestone.

HIPAA audit controls under 164.312(b) require ways to record and examine what happens in systems that hold or use electronic protected health information. The regulation does not prescribe a particular agent-container design. Identity-bound runtimes nevertheless make those records easier to attribute across the workflow, rather than leaving attribution to application code at every hop. This is where the earlier identity decision pays off.

Isolation still leaves residual risk. An agent tricked into exporting a patient list may use perfectly valid credentials, so tenant separation limits how far it can reach while prompt injection remains a risk. Human review, audit logging, and regulated infrastructure take a lot of engineering work. Dedicated environments with preconfigured applications, images, and networks meet those needs from the start of the design, before they become late additions.

Running evals and RL rollouts at scale without compromising tenant boundaries

Evals need the same disposable sandboxes as generated-code execution. Their demand simply arrives in a different shape, with many environments starting together and demand falling while the trainer updates the model. Published reinforcement-learning setups often run rollouts in CPU sandboxes while serving the model on separate GPUs. Preemption is particularly costly because an evicted rollout usually restarts from scratch. Fast provisioning alone is therefore not enough.

SCALECUA illustrates this workload with multiple rollouts per task across hundreds of parallel VM environments. Each environment needs its own boundary even when they run on a batch schedule. State leaking into a sibling sandbox corrupts benchmark results as well as exposing data. Isolation protects the validity of the experiment. Skipping it does not become cheaper merely because the agents are running an eval.

The economic requirement is rapid provisioning and teardown without idle compute costs during the trough. Many disposable sandboxes must be available together, each billed for its task and destroyed afterward. The Holistic Agent Leaderboard’s substantial rollout budget shows why evaluation cost deserves infrastructure attention.

What retrofitting isolation after launch actually costs, and why the architecture decision is effectively irreversible

Moving a shipped product from application-level separation to isolated runtimes touches more than compute. State persistence, credential handling, vector partitioning, billing, and audit attribution all depend on the existing tenant boundary. Changing one while leaving the others behind creates a period of reduced isolation. The migration is coupled work. It cannot be treated as a series of unrelated upgrades with an unchanged safety guarantee.

Lifecycle tooling adds another dependency. A pool deployment needs new per-tenant provisioning, release, and teardown mechanisms when it moves to silo, while every component touching a request needs tenant-scoped instrumentation. Axonius avoided that mismatch by connecting agent identity to its existing per-tenant authentication and deployment structure. The decision came before the agent code. Reversing it later is possible only by rebuilding the assumptions around that code.

Silo, pool, or bridge should be chosen before provisioning infrastructure, using the threat model, latency needs, and cost constraints. Compute isolation, at minimum gVisor or Firecracker, and per-tenant credential scoping belong in the first deployment. Vector namespaces and cost-attribution tags belong there too. Those choices establish where an agent can act, what it can remember, and which customer owns the result.

More in Agent Architecture