Running Firecracker on Bare Metal at Scale

Firecracker micro-VM isolation for AI agents gives each agent its own guest kernel behind a hardware virtualization boundary, while containers share the host kernel. Running Firecracker on bare metal at scale requires more than that boundary. Production depends on host configuration, resource sizing, snapshot scheduling, tenant isolation, and fleet management working together.

Why bare metal is the only viable foundation for Firecracker at production scale

Firecracker requires read/write access to /dev/kvm, the Linux kernel’s virtualization interface. Standard public-cloud VMs typically do not expose that access because they already run above a provider’s hypervisor. Firecracker’s documentation makes KVM access a prerequisite. Without it, the deployment cannot launch a microVM.

Nested virtualization offers a fallback, but it adds another virtualization layer and latency to VM operations. It also changes the isolation arrangement from direct hardware access to a boundary mediated by an outer hypervisor. Dedicated bare metal removes that outer layer. For the production architecture described here, bare metal establishes the direct KVM access on which host tuning, predictable placement, and microVM lifecycle control depend.

How do you give each AI agent its own isolated kernel without containers?

Firecracker runs each microVM with its own guest kernel using hardware virtualization. A guest-kernel exploit therefore does not, by itself, compromise a kernel shared with other tenants. Docker and runc take a different approach. Their containers share the host kernel, so a vulnerability in that kernel can expose workloads beyond the compromised container. For agents executing LLM-generated code, the distinction places the security boundary below the guest operating system.

Other runtime isolation options occupy different positions. gVisor intercepts system calls in user space, strengthening isolation over standard containers but introducing compatibility limits for some Linux software. It is not a dedicated guest kernel. Confidential computing technologies including AMD SEV-SNP and Intel TDX address the higher protection tier described for regulated workloads handling PII or healthcare data. These choices serve different requirements rather than forming interchangeable sandbox implementations.

Firecracker boots in approximately 125ms with roughly 5MB of memory overhead per microVM. That makes a separate kernel practical without traditional VM overhead. AWS built Firecracker for Lambda and Fargate, while Deno Sandbox uses Firecracker microVMs on Deno Deploy. Azure’s hardware-isolated sandboxes illustrate the broader use of virtualization boundaries, although not every provider uses the same mechanism. The relevant selection criteria are kernel separation, software compatibility, startup time, and the operational cost of keeping that isolation available.

Host configuration and the kernel settings and KVM access controls that determine fleet capacity

Production setup starts before the first guest launches. The Firecracker process needs access to /dev/kvm without running as root as a rule, often through a dedicated system user in the kvm group. The jailer is also a production requirement. Namespace isolation and seccomp-BPF filtering constrain the VMM’s host-side environment, reducing what a compromised process can reach. Guest isolation and VMM confinement protect different boundaries, so one does not replace the other.

Many small VMs can coexist only within the host’s resource limits. vm.max_map_count governs memory mappings and needs deliberate sizing for the VMM’s mapped memory. File descriptor limits need to be checked for both the process and the system because TAP devices and socket pairs use descriptors. These settings establish prerequisites for density.

Pre-allocated 2MB huge pages can improve TLB efficiency when hundreds of microVMs are resident. On multi-node NUMA hosts, placing a microVM’s cores near its memory avoids cross-NUMA latency. Storage needs the same deliberate treatment. The mq-deadline or none scheduler on NVMe drives can reduce per-operation latency for the virtio-blk devices backing root filesystems, including during bursts of snapshot writes.

MicroVM density tuning and how RAM, CPU oversubscription, and network pre-allocation interact

For the agent workloads described here, which spend much of their time idle, RAM sets the limit. The small VMM footprint does not erase guest allocations. Agents configured with 4 GiB of guest RAM each exhaust a 128 GiB host at roughly 28 resident microVMs after reserving space for the host OS and orchestration. Those allocations need to fit the target workload’s actual working set.

CPU demand follows a different pattern because agents often wait for model tokens, network fetches, or instructions. These waits allow CPU oversubscription, though the host is still RAM-bound. It does not remove the need to understand activity. Sizing depends on how many guests in the total agent population must be resident and how much CPU their overlapping active periods require.

Creation throughput also depends on work outside the VMM. Each guest needs a TAP device and a routable IP from a pre-allocated subnet pool. Setting up those resources and network namespaces when the host boots takes that work out of the microVM creation path. Disk layout has a similar provisioning role. Sparse copy-on-write overlays let guests share a base root filesystem and use separate storage for their changes, with persistent data volumes added where needed.

Snapshot-restore scheduling and the mechanism that makes idle-cost economics viable

An agent waiting for human input still takes up guest RAM if its microVM stays resident, even when it does little useful work. Snapshot-restore changes that lifecycle. The platform boots a guest to readiness, saves its state, and restores it when needed instead of retaining a fully booted VM throughout the wait. The saved state moves to storage, freeing compute capacity for the active fleet.

Firecracker’s restore mechanism makes this more than a faster reboot. It maps the snapshot memory file with MAP_PRIVATE, using lazy, page-granular copy-on-write. Restore cost thus depends on both the configured guest memory size and the working set used as the guest resumes. The guest continues from saved state. This distinction is central to agents whose installed environment and in-memory work must survive a pause.

The homelab architecture decision record reported setup-specific measurements of roughly 28ms for cold restore and 6ms for warm restore, compared with approximately 273ms for a full boot-to-ready cycle. Snapshot creation took roughly 822ms in the same measurement, but creation can happen after the agent becomes idle, outside the user-facing restore path. A guest heartbeat counter picked up at a value higher than it had before the snapshot, showing that the guest had continued from saved state.

Thousands of agents need not imply thousands of resident guests when most are waiting. Human-in-the-loop workflows can preserve kernel and memory state through pauses lasting seconds or hours, with sub-200ms wake-up as the production target described here. Storage becomes a substantial line item instead. The economic benefit depends on exchanging long idle residency for snapshot storage, while provisioning compute for active concurrency.

Multi-tenant fleet isolation across network, memory, and identity

A microVM fleet still needs rules that set boundaries for network access, stored state, and authenticated identity. Fully isolated deployments give tenants separate hosts or VM pools, while fully shared deployments rely on application-layer separation. Hybrid designs share infrastructure but separate data through namespaces or row-level security. This is the production pattern described here, with microVM boundaries protecting execution within the shared infrastructure.

Network separation begins with each sandbox’s TAP device, IP, and prepared namespace. Default-deny egress enforcement belongs at the eBPF layer rather than solely in application code, so guest behavior does not determine access to internal networks. Stored agent memory needs an independent boundary. User ID, session ID, and org ID scopes must be matched to the authenticated principal, with enforcement at the storage layer where possible. Even with a private kernel, a retrieval query can return another tenant’s data.

Tier, tenant, and user hierarchies provide a way to apply those scopes across knowledge, memory, model access, and cost tracking. Identity-scoped memory and session-scoped environments in managed agent systems illustrate this direction without establishing identical architectures across products. Accounting must follow the same tenant boundary. Recording active microVM time and snapshot storage against tenant IDs supports pricing and makes runaway workloads distinguishable from aggregate fleet growth.

Fleet management through orchestration, health monitoring, and failure recovery

Standard Kubernetes is built around OCI containers, while Firecracker requires support for its jailer, snapshot state, and restore lifecycle. Production fleets therefore need a custom control plane or purpose-built adapters to handle what ordinary container scheduling cannot. The scheduler must know which agents are resident, which are suspended, and where a restore can fit while respecting host health and NUMA placement.

A VMM can remain alive and its TAP device stay up while the guest is internally stuck. A guest-side heartbeat gives the control plane evidence from inside the isolation boundary. After restoration, continuity is also relevant. Tracking guest progress across suspension distinguishes successful resumption from a process that merely exists.

Recovery has two distinct failure domains. A host failure removes the in-memory state of every guest on that machine, requiring restoration from available snapshots on another host. A single microVM failure leaves neighboring guests running and may permit an in-place restart from a newer checkpoint when incremental snapshots are available. Maintenance uses the same lifecycle machinery deliberately. Draining a host means snapshotting active guests, moving their files to shared storage, and restoring elsewhere without losing saved agent state.

The economics of self-hosting a Firecracker fleet versus using a managed platform

Managed sandbox platforms charge for CPU and RAM while providing the API and infrastructure that launch the environment. With self-hosting, teams pay for bare-metal capacity and the work of running it in place of that service charge. Labor is the swing variable. Below break-even, maintaining the host, scheduler, isolation rules, and recovery machinery can cost more than the compute savings.

Above break-even, steady use can make self-hosting cost less. Flat-rate bare metal also makes spending more predictable for bursty agents, whose code execution may expand beyond an initially simple request. That predictability applies within provisioned capacity. Even with a fixed host bill, RAM and restore capacity have limits, and the fleet still needs enough room for agents to be active at the same time.

Snapshot scheduling improves the self-hosted model by separating total agent count from resident concurrency. It is especially relevant to per-user products where users expect persistent environments but interact intermittently. Metered idle residency works against that workload shape. The comparison must therefore cover how each platform handles suspension, what it charges for storage and active compute, and the work it takes to run a service that does the same things.

What a production-grade Firecracker platform looks like end to end

Production readiness appears in how the layers compose. A restore request must reach a host that has KVM access, enough RAM, networking set up, and the right tenant scope. The guest must then resume and demonstrate progress to the control plane. Successful boot is only one checkpoint.

A prototype can treat these transitions as manual steps. A production platform must coordinate them through idle suspension, renewed activity, host maintenance, and failure recovery. This enables the agent-native product primitive of a persistent microVM per user, with its own kernel, filesystem, installed packages, and memory surviving sleep and redeployment.

Teams choosing an operating model need to evaluate the entire lifecycle, including the sandbox launch API. Teams whose core competency is their agent product can use a managed platform built around persistent, VM-isolated agents instead of developing the hypervisor stack themselves. Teams that choose self-hosting must run every layer around it. The production threshold is a platform that can preserve state, enforce tenant boundaries, recover from failure, and make low-activity agents economical, with starting Firecracker only one requirement.