Firecracker Go SDK

The snapshotting demo demonstrates the full lifecycle of a VM snapshot: running a process inside a VM, taking a snapshot, terminating the machine, and then booting a new machine from that snapshot to verify the process state is preserved.

Prerequisites:

• KVM access
• Root access

Execution Steps:

1. Build the project: make all
2. Run the demo with root privileges while preserving your user's PATH: sudo -E env PATH=$PATH go run example_demo.go or sudo -E env PATH=$PATH make run.

What to expect:

• The VM logs and the VM's IP address will be printed to the console.
• The demo will execute a marker command (e.g., sleep 422).
• A snapshot will be created, and the machine will be terminated.
• Snapshot files will be saved in the snapshotssh folder in the current directory.
• A new machine will boot from the snapshot, and the demo will verify the marker process is still running.

To use CNI, specify a Machine with a NetworkInterface containing a CNIConfiguration.

By default, the SDK looks for CNI configuration files in /etc/cni/conf.d and CNI plugins in /opt/cni/bin. These paths can be overridden via API fields.

It is highly recommended to use CNI configuration that includes tc-redirect-tap as a chained plugin to adapt existing CNI plugins to a tap device usable by Firecracker.

Workflow:

1. Define a CNI configuration file (e.g., /etc/cni/conf.d/fcnet.conflist).
2. Ensure required plugins (like ptp, host-local, firewall, and tc-redirect-tap) are in /opt/cni/bin.
3. Configure the Machine in the Go SDK using the NetworkName (which must match the name field in your .conflist file) and the desired IfName.

{
  NetworkInterfaces: []firecracker.NetworkInterface{{
    CNIConfiguration: &firecracker.CNIConfiguration{
      NetworkName: "fcnet",
      IfName: "veth0",
    },
  }},
}

Running the full test suite requires several external resources and specific environment configurations:

Required Binaries and Files

• Firecracker and Jailer: Must be located at ./testdata/firecracker or specified via the FC_TEST_BIN environment variable. (Tested with v0.20.0).
• Linux Kernel: An uncompressed kernel binary must be located at ./testdata/vmlinux.
• Root Filesystem: Must be located at testdata/root-drive.img.
• Secondary Device Image: Must be located at testdata/drive-2.img. You can create an empty one using: dd if=/dev/zero of=testdata/drive-2.img bs=1k count=102400.

Hardware and Networking

• KVM Access: Requires read/write access to /dev/kvm and /dev/vhost-vsock.
• TAP Device: A tap device owned by your user ID must be named fc-test-tap0 or specified via the FC_TEST_TAP environment variable. To create the default tap device, run: sudo ip tuntap add fc-test-tap0 mode tap user $UID

Timeout Configuration

You can customize the test flow by setting the following environment variables:

• FIRECRACKER_GO_SDK_INIT_TIMEOUT_SECONDS
• FIRECRACKER_GO_SDK_REQUEST_TIMEOUT_MILLISECONDS

The firecracker-go-sdk uses a handler-based execution model to manage the lifecycle of a Firecracker Virtual Machine.

• Handler: A single unit of work consisting of a Name and a function Fn that accepts a context.Context and a *Machine. Handlers are used for tasks like creating network interfaces, attaching drives, or starting the VMM.
• HandlerList: A collection of Handler objects that defines a sequence of operations. You can manipulate the order of operations using methods like Append, Prepend, AppendAfter, Swap, or Remove.
• Handlers: A high-level container that categorizes handlers into two main groups: Validation (checks for configuration errors) and FcInit (the actual initialization steps for the VMM).

To execute the full initialization flow, you call Handlers.Run(ctx, m). This method flattens the validation and initialization lists into a single sequence and executes them in order. If any handler returns an error, the execution halts immediately.

// Example of manual handler manipulation
list := HandlerList{}.Append(SetupNetworkHandler, CreateMachineHandler)
list = list.AppendAfter(SetupNetworkHandler.Name, AttachDrivesHandler)

err := list.Run(ctx, machine)

A Firecracker microVM's network can be configured using either StaticConfiguration or CNIConfiguration.

Important Constraints:

• You cannot provide both CNIConfiguration and StaticConfiguration for a single NetworkInterface.
• If you are using StaticConfiguration with an IPConfiguration, you can only provide one network interface in the NetworkInterfaces slice due to limitations with the ip= kernel boot parameter.
• You cannot specify CNIConfiguration or IPConfiguration if the ip= kernel boot parameter is already provided in the kernel boot args.
• If using CNIConfiguration, it is currently only supported for VMs with a single network interface.

type NetworkInterface struct {
	StaticConfiguration *StaticNetworkConfiguration
	CNIConfiguration *CNIConfiguration
	AllowMMDS bool
	InRateLimiter *models.RateLimiter
	OutRateLimiter *models.RateLimiter
}

Use NewClient to create a new Client instance for interacting with the Firecracker API. You must provide the path to the Firecracker Unix domain socket. You can also provide a logrus.Entry for logging and a debug boolean. The client supports functional options for advanced configuration, such as mocking the operations client for testing.

To configure the request timeout, you can set the FIRECRACKER_GO_SDK_REQUEST_TIMEOUT_MILLISECONDS environment variable. The default is 500ms.

import (
	"github.com/sirupsen/logrus"
	"github.com/firecracker-microvm/firecracker-go-sdk/firecracker"
)

logger := logrus.NewEntry(logrus.New())
socketPath := "/tmp/firecracker.socket"

// Basic initialization
client := firecracker.NewClient(socketPath, logger, false)

// Initialization with options (e.g., for testing)
// client := firecracker.NewClient(socketPath, logger, false, firecracker.WithOpsClient(mockOpsClient))

To inspect the VM state manually before the snapshot is taken or after it is loaded, you can insert blocking calls (like fmt.Scanln()) into the demo code. This allows you to SSH into the VM while the program is paused.

Workflow:

1. Insert fmt.Scanln() after m.Start() to pause execution.
2. Use SSH to connect to the VM using the printed IP address: sudo ssh -i root-drive-ssh-key root@[ip]
3. Perform manual checks (e.g., ps -aux | grep "sleep 422").
4. Press Enter in the terminal running the Go program to resume execution.

err = m.Start()

// ... 

vmIP := m.Cfg.NetworkInterfaces[0].StaticConfiguration.IPConfiguration.IPAddr.IP.String()
fmt.Printf("IP of VM: %v\n", vmIP)
fmt.Scanln() // block, allows you to ssh from another shell

// ...

err = m.Start()

// ...

fmt.Println("Snapshot loaded")
fmt.Printf("IP of VM: %v\n", ipToRestore)
fmt.Scanln() // block, allows you to ssh from another shell