Firecracker microVM

Firecracker is a Virtual Machine Monitor (VMM) that uses the Linux Kernel Virtual Machine (KVM) to create and run lightweight virtual machines called microVMs.

Key Characteristics:

• Minimalist Design: Excludes unnecessary devices to reduce memory footprint and attack surface.
• Security: Combines hardware virtualization isolation with the speed of containers.
• Use Cases: Purpose-built for secure, multi-tenant container and function-based services (serverless models).
• Integration: Can be integrated into container runtimes like Kata Containers and Flintlock.

The Firecracker Jailer is a security tool designed to isolate the Firecracker process by creating a restricted environment (chroot, namespaces, cgroups, and resource limits).

Important Requirements:

• The jailer is specifically designed for Firecracker and is not intended for other binaries.
• Each jailer binary should be used with a statically linked Firecracker binary (using the default musl toolchain) of the same version.
• Experimental gnu builds are not supported.

The cpu-template-helper tool assists in creating and managing custom CPU templates for Firecracker. It provides two main command sets:

1. Template-related commands: Used to dump, strip, and verify CPU configurations.
2. Fingerprint-related commands: Used to capture host environment information and compare it against previous states to ensure template validity.

This tool is essential for providing a consistent CPU feature set to guests running on heterogeneous hardware (different CPU models).

A memory balloon device is a virtio device used to reclaim or provide guest memory via host API commands. The host sets a target size (in MiB).

• Inflation: If the actual allocated memory is smaller than the target, the device continually tries to allocate new memory in the guest.
• Deflation: If the actual size is larger than the target, the device frees memory until it hits the target.

Security Note: The balloon device is paravirtualized and relies on a guest driver. Because the guest is untrusted, a compromised driver could bypass memory restrictions. Firecracker uses madvise with MADV_DONTNEED and MAP_PRIVATE/MAP_ANONYMOUS to ensure that even if a driver is corrupted, memory cannot leak between Firecracker processes or from the host to the guest.

Firecracker is an open source Virtual Machine Monitor (VMM) designed for secure, multi-tenant, and minimal-overhead execution of container and function workloads.

Unlike QEMU, which is a general-purpose emulator, Firecracker is purpose-built for serverless functions and containers. It achieves high performance through a minimal device model and a streamlined kernel loading process, resulting in:

• Startup time: < 125 ms
• Memory footprint: < 5 MiB

Firecracker provides a RESTful control API, handles resource rate limiting, and includes a microVM metadata service for sharing configuration data between the host and guest. It only emulates 6 essential devices:

• virtio-net
• virtio-balloon
• virtio-block
• virtio-vsock
• serial console
• A minimal keyboard controller (used only to stop the microVM)

The virtio-pmem device emulates a persistent memory device (like NVDIMM) without requiring physical hardware. It is backed by a memory-mapped file on the host and exposed to the guest kernel as a region in guest physical memory. This allows the guest to access host memory pages directly via load/store instructions without needing a guest driver or VMM interaction.

From the guest user-space perspective, these devices appear as normal block devices (e.g., /dev/pmem0). This makes them suitable for use as a rootfs device to boot a VM from persistent memory.

NAT-based routing is the simplest way to connect Firecracker microVMs to the internet via the host. This method uses a Linux tap device on the host and masquerades guest traffic through the host's network interface (e.g., eth0).

Routing Approaches

1. NAT-based: Simple, but does not expose the microVM to the local network (LAN).
2. Bridge-based: Exposes the microVM to the local network.
3. Namespaced NAT: Sacrifices performance for isolation; useful when running multiple clones of the same microVM simultaneously.

Note: Firecracker currently supports only a TUN/TAP network backend with no multi-queue support.

The MMDS network stack is a minimalist implementation designed for internal microVM communication and should not be used as a security boundary.

Warning: Do not rely on the MMDS network stack to filter packets with the MMDS IP as the destination from the guest's outbound traffic. Guest traffic is untrusted.

Best Practice: Implement firewall rules at the host level to prevent guests from accessing restricted IPv4 addresses on the host. This is necessary to prevent guests from reaching host-level services like the Instance Metadata Service (IMDS).

The microVM Metadata Service (MMDS) is composed of three logical components residing within the Firecracker process but outside the KVM boundary:

1. The Backend: Part of the Firecracker API server. It handles GET, PUT, and PATCH requests from the host to manage the metadata store.
2. The Data Store: A global, thread-safe JSON-based data structure (using serde-json::Value) that holds the metadata. Its size is bounded by --mmds-size-limit.
3. Dumbo (The Network Stack): A minimalist HTTP/TCP/IPv4 stack residing in the device model. It intercepts guest network frames to respond to HTTP GET requests directed at the MMDS IP address, providing metadata to the guest without involving the host's network stack.