Sub-Second Agent Wake Latency From Snapshot

Snapshot-restore scheduling for idle AI agents replaces paid idle compute with stored machine state. A scheduler suspends an agent while it waits, then restores it on the next event. Agents have sub-second wake latency because memory is mapped on demand instead of loaded before execution, so per-user agents can respond quickly without keeping every VM running.

Why agent wake latency is an architectural property, not a tuning parameter

A developer who wants lower startup latency often turns to a warm pool or a faster boot process. Both approaches treat initialization as work that must be sped up or skipped by keeping machines resident. Snapshot-restore changes the question from how quickly a machine can boot to what restoring it actually requires. That is a different kind of operation. Cold boot runs an initialization sequence, while restore maps saved memory and unpauses vCPUs without replaying initialization.

The work removed from the critical path does not depend on shaving time from each startup service. It has already happened before the snapshot was taken. Product teams can therefore check how restore itself works instead of taking an unusually fast boot benchmark as proof that the infrastructure will respond quickly every time.

What snapshot-restore actually does at the hardware level

A frozen microVM already has a running kernel, a warm page cache, and active processes. Restore does not run init, re-probe devices, or start systemd. Firecracker maps vm.mem through MAP_PRIVATE, a lazy, copy-on-write memory mapping, instead of reading the entire RAM file into resident memory first. It then unpauses the vCPUs, with guest pages faulting in as execution proceeds. The guest resumes before all its memory arrives.

Demand paging is the load-bearing mechanism here. MAP_PRIVATE, with optional userfaultfd support, lets the restore path defer memory loading until execution needs particular pages. ForgeVM describes the resulting continuity as a guest returning to the point at which it was frozen, with its agent already running. The saved machine does not come from a warm pool, which would need to keep resources resident all the time. Nor is it a container restart that shares the host kernel and replays process initialization.

The QEMU q35 and virsh stack described in the restore benchmark follows a different path. It copies the full RAM image rather than mapping it on demand, so restoring a warm idle guest takes several seconds, with a lightly loaded agent VM taking several seconds more. This is a structural difference.

How the snapshot pipeline converts a one-time cold boot into every subsequent fast restore

The snapshot lifecycle separates a one-time bake from repeated on-demand restores. During the bake, the template microVM boots until its kernel and agent process are ready and its initialization sequence is done. The runtime pauses it and serializes RAM, CPU, and device state to disk. Only the restores that follow are on the user-facing critical path. ForgeVM describes each spawn as making a copy-on-write snapshot copy, starting a new Firecracker process with --restore-from-snapshot, and resuming the ready guest.

The base snapshot remains read-only across those spawns. Each VM gets copy-on-write overlays for its memory file and root filesystem, so reads fall through to the base while writes stay private. Destroying a VM deletes its overlay without changing the template. Many VMs can consequently share unchanged memory pages, with only the pages they write becoming unique.

The surrounding steps must also remain cheap. PandaStack keeps only pre-built network plumbing warm, including a network namespace, veth pair, tap, and iptables configuration. Setting those up on demand would add ~100ms to the cold path. The bake is also a security boundary because the RAM it captures may hold credentials. The recommended lifecycle captures a pre-credential clean base and injects secrets during restore through a separate enrollment path.

What the latency numbers actually measure and what they leave out

A restore benchmark measures only part of what an agent product experiences. Five distinct clocks separate backend-ready, when node-local restore returns, from resume RPC, service-ready, time to first command, and time to useful work. Resume RPC includes time spent on routing and control-plane work, while service-ready is when the guest agent accepts work. Time to useful work ends when a useful-work oracle validates the result. VM resume is only the first clock.

Kimi K3 AgentENV, from Moonshot AI and kvcache-ai, combines Firecracker, OverlayBD, ublk, and incremental memory and filesystem snapshots. AgentENV reports snapshot-backed boot or resume below 50ms and pause below 100ms. Those measurements do not traverse the full client, HTTP gateway, scheduler, placement path, or persistent metadata layer. They also do not include running a user command or returning output to the policy. A fast primitive is necessary, but not sufficient.

The complete agent step adds LLM action decode, sandbox acquire, tool execution, observation return, and next-LLM scheduling. As resume becomes a well-engineered primitive, the serialized boundary between model generation, environment execution, and the next useful observation becomes the next bottleneck. Measurements on a quiet node also miss what happens when the fleet is busy. Synchronized-barrier workloads expose herd behavior, while open-loop arrivals reveal queue accumulation. The lesson for developers is to ask for p99 under concurrent restore pressure alongside p50 from serial runs.

Why snapshot-restore makes per-user persistent agents economically viable

Agents spend much of their time off CPU, waiting for LLM inference, user input, or tool responses. A fleet of thousands of agents can therefore be mostly idle at any given moment. A warm pool does not turn that inactivity into savings. Keeping every VM resident means paying all the time for enough RAM and CPU to meet peak demand. When CPU utilization falls below a certain level, memory becomes the biggest VM cost.

Hibernate-on-wait scheduling converts those waiting periods into opportunities to suspend compute. When an agent waits on inference or user input, the scheduler snapshots and suspends it, then restores it on the next event. Because the runtime freezes a running machine instead of shutting it down, processes resume mid-execution and open files stay open. Files and process state do not have to be reconstructed for each task.

A delay of seconds on repeated wakes makes aggressive hibernation incompatible with a responsive product, while sub-second restore makes paying for active concurrency practical. The economic distinction is between idle RAM and snapshot storage. This is what makes a dedicated VM per user potentially cost-competitive with shared infrastructure, rather than requiring a continuously running machine for every account. The savings depend on scheduling and billing as well as the restore primitive.

What snapshot-restore enables beyond fast wake: checkpoint, rollback, and tree search

Once restore is cheap enough for repeated use within a task, it becomes a state-management primitive. An agent can checkpoint an execution state, explore branches, and roll back instead of rebuilding its environment after each attempt. This also lets agents run test-time tree search, multi-branch reinforcement learning rollouts, and speculative execution. Fast wake becomes more than a responsiveness feature. Branching requires preserving running state, executing alternatives, and discarding unsuccessful paths, which a warm pool alone does not provide.

DeltaBox, from Shanghai Jiao Tong University and Huawei, was published on arXiv in 2026. It builds DeltaFS and DeltaCR on Firecracker microVMs and reports 14ms checkpoint and 5ms rollback, enabling exploration of more nodes within fixed time budgets in software engineering benchmarks and reinforcement learning micro-benchmarks. Its fork-based restore mechanism duplicates page tables without copying memory contents. A large agent state need not impose the full-memory-copy delay on each branch.

AgentENV applies incremental memory and filesystem snapshots to large-scale agentic reinforcement learning, where agents branch, evaluate, and roll back millions of times. Its sub-50ms restore is a throughput requirement for that loop. Incremental checkpointing tracks what has changed instead of saving all RAM at every pause, so frequent mid-task saves do not take up most of the step time. For product developers, the same architecture lets agents wake quickly from idle and explore saved state over and over.

The isolation boundary that makes snapshot-restore safe for multi-tenant agent fleets

An agent that executes arbitrary code needs an isolation boundary independent of its snapshot mechanism. A container shares the host kernel, so a kernel vulnerability can expose other containers on that host. A microVM instead places a hardware virtualization boundary, through KVM, between the guest and host. Each guest has its own kernel. This is the stronger primitive for agents that install packages, run arbitrary binaries, and modify their filesystems without granting access to neighboring workloads.

Firecracker reduces the exposed device surface deliberately. Its device model includes virtio-block, virtio-net, virtio-vsock, virtio-rng, a serial console, and a one-button keyboard, but no USB, GPU, or PCI bus. With fewer emulated devices, guest-controlled input has fewer paths to the host. That does not make the boundary impenetrable. How strong that boundary is still depends on the host kernel code that handles guest data.

Vercel reported a guest-to-host root escape in October 2026, paid its maximum $50,000 bounty, and identified the issue through its CEO as a KVM zero-day. MicroVM escapes need hypervisor-level vulnerabilities and are harder to chain than shared-kernel container escapes, but keeping the host kernel secure is still essential. Snapshot hygiene addresses a separate risk. The clean-base enrollment design keeps credentials from being left in copy-on-write forks across tenants, while KVM provides execution isolation.

The operational constraints developers encounter when building on snapshot-restore

Firecracker snapshot-restore is not a drop-in replacement for Docker. Networking, block devices, and snapshot lifecycle management require explicit decisions that Docker abstracts away. These are known costs with known solutions. Consistent sub-200ms restores require block-level image caching rather than putting docker pull on the microVM fast path. The bake must also be integrated into the image build pipeline so that a ready snapshot exists before the first restore request.

PandaStack uses XFS reflink cloning for O(metadata) copy-on-write root filesystem copies. DeltaBox uses DeltaFS, a custom Linux overlayfs extension, with an XFS root filesystem block device for layered state. Runtime compatibility imposes further constraints. On the QEMU/libvirt stacks described in the notes, virtiofs through vhost-user blocks virsh save, managedsave, and migration at the protocol level, requiring detachment before snapshot or a suitable runtime change.

Firmware can constrain the available snapshot path too. UEFI/pflash blocks internal QEMU snapshots with legacy HMP savevm or raw-format nvram, while newer QEMU/libvirt configurations, including libvirt 10.9.0+, lift that restriction with qcow2-backed nvram and the new QMP snapshot commands. These details help explain the appeal of a minimal device model. It takes several engineering-months to build bake automation, network slot pools, copy-on-write storage, and secrets enrollment. Teams building agent products have good reason to choose infrastructure where that work is already done.

What to look for in agent infrastructure that takes snapshot-restore seriously

For developers evaluating infrastructure, the question is whether its architecture produces the advertised wake time. Demand-paged restore through userfaultfd or an equivalent mechanism avoids making full RAM loading a prerequisite for vCPU resume. Per-VM kernels provide the appropriate isolation primitive for untrusted execution, subject to the host security limits described above. Persistence must cover files, credentials, and process state across sleep and redeploy. A restart that reconstructs an environment is not equivalent.

With hibernate-on-wait, the bill must switch from idle compute charges to storage charges for fast restore to bring down the cost per user. Shared templates must stay pre-credential, with secrets added to each fork through enrollment. End-to-end p99 measurements must cover the path from API call to a guest ready for commands, under concurrent pressure.