Firecracker Snapshot and Restore for Agent State Preservation

Firecracker snapshot-restore scheduling lets idle AI agents release compute, retain machine state in storage, and resume when work arrives. It lets each user have a persistent agent without keeping a VM running for that user all the time. That guarantee depends on preserving the separate rootfs disk with the snapshot and implementing an explicit resume contract for the world outside the guest.

What a Firecracker snapshot actually captures and why that matters for agents

A Firecracker snapshot freezes guest RAM, CPU register state, and emulated device state at one instant. The rootfs disk is a separate file that must remain in lockstep with the captured machine state. That distinction defines the boundary of preservation. A memory image without its corresponding disk is not the complete working environment an agent needs to continue.

On restore, Firecracker maps the saved memory, loads CPU and device state, and resumes the guest’s execution from the instruction where the VM paused. The guest continues rather than reconstructs its work. It has no inherent awareness that execution stopped, even though the outside world may have advanced during the pause.

This preserves something different from semantic memory in vector databases or RAG indexes. Those systems retain what an agent knows, while a snapshot preserves the machine on which it is working. Retrieved facts cannot resume a half-finished process or restore its live memory. When the matching rootfs is kept, snapshot-restore can resume the whole machine with close to zero loss, without rebuilding it from selected application variables.

How copy-on-write restore makes snapshots cheap enough to use at fleet scale

Restore is inexpensive because it does not require eagerly copying the entire memory image into a new VM. Firecracker maps snapshot memory with MAP_PRIVATE, allowing restored instances to share read-only pages until a guest writes to them. The kernel then copies only the changed page.

VMs restored from the same base can share most of their memory, while pages dirtied after restore become private to each instance. The rootfs follows a corresponding copy-on-write pattern, with a read-only base and a separate writable overlay for each VM. Destroying an instance removes its overlay without changing the reusable base.

There is no warm pool behind this mechanism. Each restore is a cold snapshot bring-up. The first cold boot of a template takes on the order of seconds, but later instances bypass kernel boot and agent startup. Template preparation becomes an amortized cost, although capturing an individual agent’s changing state remains a separate lifecycle expense.

How do you stop paying for compute while agents sit idle?

Agent tool sandboxes have sharp CPU and memory spikes during execution but spend most of their trajectories at low utilization. Waiting for human approval or a webhook can stretch those quiet periods to minutes or hours. A resident VM consumes compute throughout that interval. For fleets with this workload shape, waiting becomes the dominant compute cost.

Keeping every sandbox resident preserves continuity but charges for the entire wait. Destroying it removes that charge but imposes a rebuild penalty when the next request arrives. Neither strategy provides both properties.

Snapshot-restore provides a third option by hibernating the agent, releasing compute, and retaining its state in storage until the next message arrives. A coding agent can resume without rebooting or re-cloning its repository, while a research agent can retain its downloaded corpus and warm caches overnight. Idle compute becomes idle storage instead. The remaining scheduling trade-off is keepalive duration, since a shorter window causes more capture-restore cycles, while a longer one retains the running sandbox between nearby requests.

Snapshot-restore as the infrastructure primitive for per-user agent deployment

A per-user agent needs to start quickly, keep its state between sessions, and cost little enough while idle to serve a large user base. Fast restore and storage-only dormancy meet those needs together. The deployment unit can become the user’s machine. Its lifetime no longer has to match either a single request or a permanently allocated VM.

Without fast stateful suspend, a warm pool can reduce startup delay but retains idle capacity, while cold boot avoids that reservation at the expense of responsiveness. Neither directly supplies a persistent working environment that can disappear from compute between sessions. Snapshot-restore changes that architectural choice. Per-session microVM patterns, including silo, pool, and bridge arrangements, can preserve dedicated runtime boundaries without requiring every inactive runtime to remain resident.

Memory flexibility supports the same deployment model. Firecracker supports memory hotplugging without downtime, allowing a restored agent to grow into its session’s needs rather than requiring its full possible allocation at spawn time. This is useful when tool execution has unpredictable memory demands.

What production implementations show about restore latency and the snapshot lifecycle

Production implementations have each reached sub-50 ms restore times, showing that they can bring an agent back quickly with its state intact. ForgeVM reports substantial improvements in both single and concurrent sandbox spawns, with snapshot-restore contained inside its Firecracker provider and no changes required elsewhere in the stack. Transparency is an important part of that result. The application can retain its execution model while the provider changes how the machine becomes ready.

PandaStack restores every sandbox from a baked Firecracker snapshot without a warm pool. Its create-to-ready measurements stay fast as it creates one sandbox after another from a base-template, and its end-to-end snapshot creation requests have low median and tail latency. Only the template’s initial cold boot takes on the order of seconds. This separates the cost of preparing a reusable environment from the recurring cost of making an instance available.

The homelab ADR 022 architecture uses a Kubernetes controller to manage the snapshot lifecycle and measures boot-to-ready, snapshot creation, and restore across cold and warm paths. Its guest heartbeat counter resumed above its pre-snapshot value, showing that the guest had continued without a reboot. Snapshot creation still takes real time. That cost informs when capture is worthwhile, even when the subsequent restore is fast.

A full snapshot consumes roughly one guest-RAM of disk space, so documented mitigations include right-sizing memory, dirty-page diff snapshots, thin-pool storage, TTL or idle eviction, and local NVMe per host. Capture timing has a separate correctness requirement. An in-VM supervisor can observe that no model or tool call is in flight, signal safe quiescence to the controller, and re-establish connections after resume.

Fork-and-branch — snapshot-restore as a parallelism primitive, not just a cost tool

The same snapshot can restore into multiple independent VMs at once, turning a saved execution point into a branch point. The shared work is already complete. Only execution after the branch diverges, allowing a successful path to be promoted without rebuilding the starting environment for every attempt.

This supports speculative tool execution, multi-path planning when the best route is unknown, and adversarial testing in which one branch tries to break what another built. Each branch gets the parent’s captured machine state, and the changes it makes later stay separate from those of the other children. Snapshot-restore thus serves both temporal suspension, by resuming later, and spatial expansion, by branching now.

Copy-on-write limits the resources needed for that expansion. Branches share the parent’s unchanged memory pages and base disk, allocating private state as they write. Branching is therefore not equivalent to allocating a full independent copy of the starting environment for every path.

The correctness problems that survive a restore — what the guest cannot see

Faithful machine capture does not freeze the environment around it. CPU feature detection, for example, runs at boot and leaves cached function pointers and code-path choices in guest memory. A restore on a host that lacks a required instruction may seem to work, then cause a SIGILL fault when execution reaches that path. The direction of portability is asymmetric. Baking on a lesser CPU and restoring on a richer one is generally safe, while the reverse requires a CPU template that normalizes CPUID to a fleet-wide baseline at bake time.

Restore detection has a similar blind spot because the snapshot also saves the evidence inside the guest. A saved boot_id or generation counter cannot distinguish clones restored from the same snapshot. Firecracker’s random-for-clones documentation says that those clones read the same boot_id, and the zylos-ai research PR showed how this caused an in-guest detection scheme to fail. The signal must come from outside the snapshot. A value that the host or provider updates on every restore gives the guest that signal, so resume handling can include entropy reseeding to keep cloned pools from producing identical random streams.

External connections also outlive the guest’s knowledge of them. A saved TCP socket may no longer have a live peer after an idle timeout, while credentials and leases may expire during dormancy. Resume must explicitly re-establish connections and revalidate credentials and leases.

A request accepted before capture may finish while the agent sleeps, so the restored process may not know how it turned out. Blind replay can duplicate the action. Idempotency and reconciliation are therefore required for operations that may settle during dormancy. These are engineering problems with documented solutions, but the contracts around restoration must implement those solutions explicitly.

What snapshot-restore requires from the infrastructure layer that hosts it

The hosting platform must make these properties hold up beyond a fast restore demonstration. Its scheduler must expose workload-specific keepalive policy governing when each agent sleeps. A frequent interruption pattern differs from an overnight pause. Configurable scheduling lets operators balance capture frequency against resident time using the behavior of their actual sessions.

Snapshot storage must be durable, fast to read, and governed by an explicit retention policy. Mapping the memory file is only useful if its pages can be supplied quickly when execution needs them. Lifecycle management is equally necessary. Diff-snapshot support reduces repeated full-image storage, while TTL and idle eviction prevent inactive sessions from accumulating without bound.

The platform also owns the external restore token. It must update and deliver that value on every restore because the guest cannot reliably manufacture evidence of an event that reset its own detection state. Without it, entropy reseeding and connection re-establishment cannot depend on reliable detection of each restoration.

Per-agent microVM isolation gives each agent its own kernel and captured state, rather than a shared-kernel environment whose state may be affected by co-tenants. Firecracker relies on KVM and hardware virtualization extensions. The infrastructure must therefore provide bare-metal or KVM-passthrough hosts, since a cloud VM without exposed KVM cannot support the stated production configuration. A platform meeting these requirements offers more than a pause button, making stateful suspension a schedulable operation with explicit storage, isolation, and resume guarantees.