Apple Containerization

repository·main·Indexed 27 days ago

https://github.com/apple/containerization

A Swift-based library for running Linux containers on Apple silicon using Virtualization.framework or Linux KVM/cloud-hypervisor. It provides low-level APIs for OCI image management, ext4 filesystem creation, lightweight VM orchestration, and process management. The library supports running linux/amd64 containers on Apple silicon via Rosetta 2 and includes specialized toolsets like ContainerizationEXT4 and a CloudHypervisor client for VMM and VM lifecycle management.

Tokens
3.7K
Snippets
7
Records
23
Agent score
94%

What's inside apple-containerization

  1. Overview of Containerization APIs

    main

    The Containerization package allows applications to use Linux containers by executing them inside lightweight virtual machines. It provides Swift APIs for several core container operations:

    • OCI Image Management: Manage OCI images and interact with remote registries.
    • Filesystem Operations: Create and populate ext4 file systems.
    • Networking: Interact with the Netlink socket family.
    • Kernel Management: Create optimized Linux kernels for fast boot times.
    • Runtime Management: Spawn lightweight virtual machines and manage the runtime environment.
    • Process Management: Spawn and interact with containerized processes.
    • Architecture Support: Use Rosetta 2 for running linux/amd64 containers on Apple silicon.

    On macOS, it uses Virtualization.framework. On Linux, it uses cloud-hypervisor + KVM.

  2. CloudHypervisor Compatibility and Requirements

    main

    Requirements

    • Platform: The library compiles on macOS and Linux, but the cloud-hypervisor binary itself only runs on Linux.
    • Dependencies: Requires swift-nio and swift-log.

    Compatibility

    • Target API: /api/v1/ REST namespace.
    • Minimum Version: Tested against cloud-hypervisor v40 and later. Earlier versions may have incompatible JSON schemas or missing endpoints.
  3. Prerequisites for x86_64 deployment builds

    main

    Before running make dist-x86_64, ensure the following are configured:

    1. Pinned Source Checkouts: Manually clone required dependencies into the .local/ directory. The build does not fetch them automatically.
      git clone -b v52.0 https://github.com/cloud-hypervisor/cloud-hypervisor .local/cloud-hypervisor
      git clone https://gitlab.com/virtio-fs/virtiofsd .local/virtiofsd
    2. x86_64 Kernel: An x86_64 kernel must exist at kernel/vmlinuz-x86_64 or kernel/vmlinux-x86_64. You can build it using:
      make -C kernel TARGET_ARCH=x86_64
    3. Linux Dev Image: The build depends on the linux-image target. The first run will automatically handle the container build cache for the dev image.
    git clone -b v52.0 https://github.com/cloud-hypervisor/cloud-hypervisor .local/cloud-hypervisor
    git clone https://gitlab.com/virtio-fs/virtiofsd .local/virtiofsd
    
    make -C kernel TARGET_ARCH=x86_64
  4. Initialize a CloudHypervisor Client

    main

    To interact with the cloud-hypervisor REST API over a Unix domain socket, instantiate a CloudHypervisor.Client. You must provide a socketPath as a URL pointing to the .sock file.

    By default, the client manages its own MultiThreadedEventLoopGroup and shuts it down on deinit. If you want to provide your own eventLoopGroup (e.g., to share resources with other NIO-based libraries), pass it via the eventLoopGroup: parameter. Note that if you provide your own group, the client will not shut it down on deinit.

    import CloudHypervisor
    import NIOPosix
    
    let group = MultiThreadedEventLoopGroup(numberOfThreads: 2)
    defer { try? group.syncShutdownGracefully() }
    
    let client = try CloudHypervisor.Client(
        socketPath: URL(filePath: "/run/ch/vm0.sock"),
        eventLoopGroup: group
    )
  5. Use a pre-built Linux kernel

    main

    You can provide your own kernel to Containerization. If using a pre-built kernel, ensure it has VIRTIO drivers compiled directly into the kernel (not as modules).

    Recommended Source: The Kata Containers project provides an optimized Linux kernel. Artifacts can be found on their releases page, or the image vmlinux.container can be located at /opt/kata/share/kata-containers/.

  6. Generate Protobufs and API Documentation

    main

    Containerization uses grpc-swift and swift-protobuf. Use the following commands for maintenance tasks:

    Regenerate RPC interfaces:

    make protos

    Generate and view local API documentation:

    make docs
    make serve-docs

    After running make serve-docs, preview the documentation at http://localhost:8000/containerization/documentation/.

    make protos
    
    make docs
    make serve-docs
    # Then open http://localhost:8000/containerization/documentation/
  7. Configure the Linux backend for Containerization

    main

    When using the Linux backend (CHVirtualMachineManager), the following requirements must be met on the host:

    • Binaries: cloud-hypervisor and virtiofsd must be available on PATH. You can provide explicit URLs to CHVirtualMachineManager.init to override default lookups.
    • KVM Access: The calling user must have read and write access to /dev/kvm.
    • Networking: You are responsible for pre-staging TAP, bridge, or NAT plumbing. The TAPInterface type consumes an existing TAP device by name.
    • Storage: Uses virtio-blk for block storage and virtio-fs (via virtiofsd) for shared directories.
  8. Set up the build environment for Containerization

    main

    To build the Containerization package on macOS, ensure you meet these requirements:

    Hardware/Software Requirements:

    • Mac with Apple silicon
    • macOS 26
    • Xcode 26

    Setup Steps:

    1. Set the active developer directory to your installed Xcode:
    sudo xcode-select -s <PATH_TO_XCODE>
    1. Install the container CLI (from apple/container) and ensure it is on your PATH. This is used to compile the Linux guest (vminitd/vmexec) inside a Linux container, so you do not need a Swift toolchain or Static Linux SDK on your host.
    container --version
    sudo xcode-select -s <PATH_TO_XCODE>
    
    # Install per https://github.com/apple/container, then verify it is on PATH:
    container --version
  9. Understand Single File Mounts behavior

    main

    In Containerization, virtiofs only supports sharing directories. To support mounting a single file from the host into a container, the system uses a multi-step process:

    1. Detection: The system detects if a mount source is a regular file (resolving symlinks first).
    2. Parent directory share: The file's parent directory is shared via virtiofs into the guest VM. If multiple files from the same directory are requested, only one virtiofs share is created.
    3. Guest holding mount: The parent directory is mounted to a holding location within the guest VM.
    4. Bind mount: The specific file is then bind-mounted from the guest's holding location to the requested destination path inside the container.

    Note on visibility: Because the parent directory is shared via virtiofs, sibling files in that directory are visible to the guest VM at the holding mount point (under /run), even though the container itself only sees the specific file requested.

  10. Build an x86_64 Linux deployment tarball

    main

    Use make dist-x86_64 to produce a self-contained x86_64 Linux deployment tarball located at bin/containerization-x86_64-<sha>.tar.gz. This build runs inside the aarch64 Linux dev container and requires make and container on the host.

    The resulting tarball contains:

    • cctl host binary (static musl)
    • cloud-hypervisor VMM (static musl)
    • virtiofsd filesystem daemon (dynamic glibc)
    • An x86_64 Linux kernel
    • initfs.ext4 guest rootfs (containing vminitd + vmexec)

    Deployment Host Requirements: Because virtiofsd is dynamically linked, the target host must provide:

    • glibc ≥ 2.35 (e.g., Ubuntu 22.04, Debian 12, RHEL 9)
    • libseccomp.so.2
    • libcap-ng.so.0
    make dist-x86_64
  11. Select a target architecture for the kernel build

    main

    The build target is controlled by the TARGET_ARCH make variable. If unset, the build defaults to the host architecture (normalizing aarch64 to arm64 and amd64 to x86_64).

    Available Architectures:

    • arm64: Produces vmlinux-arm64 (an uncompressed Image).
    • x86_64: Produces vmlinuz-x86_64 (a compressed bzImage). This is cross-compiled inside the arm64 container.

    Commands:

    • Build for host: make
    • Build for arm64: make TARGET_ARCH=arm64
    • Build for x86_64: make TARGET_ARCH=x86_64 or the alias make x86_64.

    Resulting kernels are placed in the bin/ directory.