Sysbox Documentation

repository·master·Indexed 26 days ago

https://github.com/nestybox/sysbox

Sysbox is an open-source container runtime (a specialized runc) that enables containers to run system-level software such as systemd, Docker, and Kubernetes without the overhead of nested virtualization. It provides hardened isolation using Linux user-namespaces and virtualizes procfs and sysfs to create a VM-like environment. The documentation covers installation via packages or source, deployment patterns for Docker-in-Docker (DinD) and Kubernetes-in-Docker (KinD), and configuration for Docker and Kubernetes using the sysbox-runc runtime.

Tokens
28.6K
Snippets
68
Records
200
Agent score
85%

What's inside Sysbox

  1. Overview of Sysbox container runtime

    master

    Sysbox is an open-source container runtime (a specialized runc) that enhances container isolation and enables containers to run system-level workloads typically reserved for VMs.

    Key Capabilities:

    • Improved Isolation: Uses Linux user-namespaces so the container's root user has zero privileges on the host. It virtualizes portions of procfs and sysfs and hides host information.
    • System Workloads: Allows running software like systemd, Docker, Kubernetes, K3s, and buildx inside containers without modification, special images, or privileged mode.
    • VM-like Environment: Provides a VM-like experience without the overhead of actual VMs or the need for nested virtualization in cloud environments.
    • Coexistence: Can run side-by-side with other runtimes like the default runc or Kata.
  2. Understand Sysbox container isolation and namespaces

    master

    Sysbox system containers provide enhanced isolation by always using all Linux namespaces. Unlike standard Docker containers (which rely on the container manager to choose namespaces), Sysbox automatically enables all namespaces, including cgroup and user namespaces, to ensure stronger isolation.

    Namespace Comparison:

    NamespaceDocker + SysboxDocker + OCI runc
    mountYesYes
    pidYesYes
    utsYesYes
    netYesYes
    ipcYesYes
    cgroupYesNo
    userYesNo (unless configured)
  3. Understand System Container Images

    master
    System container images are standard Docker images. While they often contain system software like systemd, docker, or k8s, they can also carry application software. You do not need specialized entrypoints or complex docker run commands to deploy them because the Sysbox runtime handles the necessary configuration to ensure the software runs correctly inside the container.
  4. Distinguish between Inner and Outer Containers

    master

    When using nested container environments (like running Docker inside a system container), Sysbox uses specific terminology to avoid confusion:

    • Outer Container: The system container created at the host level, launched using Docker and Sysbox.
    • Inner Container: The application container created within the outer container (e.g., a container managed by the Docker or Kubernetes instance running inside the system container).
  5. Understand Initial Mount Immutability

    master

    Sysbox implements "immutable mounts" for filesystem mounts established during container creation (the rootfs jail). This prevents processes inside the container—even those running as root with full capabilities—from modifying these specific mounts in ways that would weaken container isolation.

    Restrictions apply to both the container's primary mount namespace and any nested namespaces created via unshare -m.

    Default Restrictions:

    • Remounts:
      • A read-only immutable mount cannot be remounted as read-write.
      • Attributes of immutable mounts (e.g., nosuid, noexec, relatime) cannot be changed via remount.
    • Unmounts:
      • The root filesystem (/) cannot be unmounted.
      • The /proc and /sys mounts (and their submounts) cannot be unmounted.
      • Other immutable mounts can be unmounted (to allow process managers like systemd to clean up during container stop), which may expose the underlying container image contents.

    Restricted operations will fail with EPERM (permission denied).

  6. Access Sysbox documentation

    master

    Sysbox documentation is organized into several specialized guides depending on your needs:

  7. Understand Sysbox System Containers

    master
    Sysbox creates system containers, which are designed to act as lightweight, efficient "virtual hosts" or "containers of containers." Unlike traditional application containers that package a single micro-service, system containers can bundle multiple services such as Systemd, sshd, Docker, or Kubernetes. They provide a more flexible and portable alternative to Virtual Machines (VMs) with strong isolation, allowing you to run services and launch inner containers just as you would on a physical host.
  8. Understand Sysbox Architecture and Components

    master

    Sysbox consists of three main components that work together to provide system containers. Users typically interact with these via higher-level tools like Docker or containerd rather than directly.

    • sysbox-runc: The container runtime (front-end) that performs low-level kernel setup. It is invoked by Docker/containerd to launch containers.
    • sysbox-fs: A FUSE daemon (back-end) that emulates portions of procfs and sysfs to make the container resemble a virtual host while maintaining isolation.
    • sysbox-mgr: A daemon (back-end) that manages user-ID/group-ID mappings and special mounts. Communication between components occurs via gRPC.
  9. Understand Sysbox system call interception

    master
    Starting from version v0.2.0+, Sysbox performs selective interception of "control-path" system calls, specifically targeting calls like mount and umount2. This interception is used to enable proper procfs and sysfs virtualization while minimizing performance impact on processes running inside the system container.
  10. Virtualization of procfs and sysfs

    master

    Sysbox performs partial virtualization of procfs and sysfs to allow system containers to function like real machines while protecting the host.

    • Procfs Virtualization: Exposes /proc as read-write so container processes can interact with it, but prevents them from modifying system-wide host settings. This virtualization is independent for each system container and extends to any inner containers.
    • Sysfs Virtualization: Protects host settings via sysfs. Additionally, /sys/fs/cgroup is mounted read-write to allow container processes to manage a subset of cgroup resources assigned to the container itself.
  11. Share storage between Sysbox containers

    master
    To share storage between multiple Sysbox containers, mount the same storage source to each container. Even if the containers use different user-namespace ID mappings, Sysbox uses ID-mapped mounts or shiftfs to ensure all containers see the storage with a consistent set of filesystem user-ID and group-IDs.