runc

The container's lifecycle involves a synchronization step between the parent process and the container's init process:

1. Synchronization: The parent creates a pipe and passes it to the container's init via FD 3. The init process blocks on this pipe until the parent closes its side. This allows the parent to finish setting up cgroups and user namespace mappings (UID/GID) before the application runs.
2. Execution: The application consuming libcontainer should be compiled statically. The runtime uses exec to start the process inside the container. There should be no long-running init process within the container spec.
3. I/O Setup: If a pseudo TTY is provided, the runtime opens and dup2s the console as the container's STDIN, STDOUT, and STDERR, and mounts the console at /dev/console.
4. Runtime Files: The runtime populates extra files such as /etc/hosts, /etc/resolv.conf, /etc/hostname, and /etc/localtime.

The name of the systemd unit and its containing slice are derived from the Linux.CgroupsPath field in the container runtime spec (config.json).

Using Linux.CgroupsPath

If Linux.CgroupsPath is set, it must follow the format [slice]:[prefix]:[name]:

• slice: The systemd slice under which the container is placed.
  • Defaults to system.slice.
  • If using cgroup v2 with a rootless container, it defaults to user.slice.
  • Use - to represent a root slice.
  • Sub-slices are denoted with dashes (e.g., user-1000.slice).
  • Note: Slashes (e.g., user.slice/user-1000.slice) are invalid.
• prefix and name: Used to compose the unit name as <prefix>-<name>.scope.
  • If name has a .slice suffix, the prefix is ignored and the name is used as is.
  • Default values for both are empty strings.

Default Behavior

If Linux.CgroupsPath is not set or is empty, runc defaults to the equivalent of :runc:<container-id>.

The nsenter package provides a special init constructor via Cgo that executes nsexec() before the Go runtime boots. This is necessary to call setns(2) and clone(2) to join Linux namespaces without encountering issues caused by the Go runtime's multi-threaded nature.

To ensure the nsexec() constructor is executed when your application starts, you must perform a blank import of the package. If you are using libcontainer directly and do not import nsenter, the re-exec process will fail to join the required namespaces.

Lifecycle of nsexec():

1. It retrieves the init pipe file descriptor from the _LIBCONTAINER_INITPIPE environment variable.
2. It reads bootstrap data (namespace paths, clone flags, UID/GID mappings, and console path) from the pipe.
3. It calls setns(2) to join existing namespaces.
4. It calls clone(2) with the provided flags (this is required even without CLONE_NEW* flags to allow entering the PID namespace).
5. It updates UID/GID mappings and performs miscellaneous setup.
6. It sends the child PID back to the parent and exits, allowing the Go runtime to take over in the newly joined namespaces.

import _ "github.com/opencontainers/runc/libcontainer/nsenter"

Cgroups are used to manage system resources like CPU, memory, and device access. The following subsystems are enabled by default in v1 containers:

Synchronization Note: The parent process of the container's init must place the init PID into the correct cgroups before initialization begins. This is synchronized via a pipe (passed to the init process via FD 3) to ensure no processes escape the cgroups during setup.

| Subsystem  | Enabled |
| ---------- | ------- |
| devices    | 1       |
| memory     | 1       |
| cpu        | 1       |
| cpuacct    | 1       |
| cpuset     | 1       |
| blkio      | 1       |
| perf_event | 1       |
| freezer    | 1       |
| hugetlb    | 1       |
| pids       | 1       |

A container requires a root filesystem (rootfs) to jail and spawn processes. The runtime automatically sets up several essential filesystems within the container's mount namespace.

Key Filesystem Mounts:

Device Nodes: After mounting filesystems, the runtime populates /dev with specific device nodes. A standard rootfs does not need to include these, as the runtime provides them:

Standard Symlinks: Once /proc is mounted, the following symlinks are established for I/O:

• /dev/fd $\rightarrow$ /proc/self/fd
• /dev/stdin $\rightarrow$ /proc/self/fd/0
• /dev/stdout $\rightarrow$ /proc/self/fd/1
• /dev/stderr $\rightarrow$ /proc/self/fd/2

Root Jailing: The runtime uses pivot_root to jail the process inside the rootfs. If the rootfs is inside a ramfs, MS_MOVE combined with chroot is used instead, as pivot_root is not supported in ramfs.

runc operates in two primary execution modes. Choosing the right one depends on whether you are using the CLI interactively or building a high-level container manager.

Foreground Mode (Default)

In this mode, the runc process remains running as a parent to the container process.

• Behavior: All stdio is buffered through the foreground runc process.
• Pros: Most straightforward for interactive shell use; handles stdio management for you.
• Cons: Requires a long-running runc process. If the runc process is killed (e.g., via OOM-kill or manual signal), the container will likely die due to SIGPIPE. It also prevents passing file descriptors directly to the container (unlike --preserve-fds).

Detached Mode

In this mode, runc exits once the container has started, leaving no long-running runc process.

• Behavior: The caller is responsible for managing the container's stdio and acting as a subreaper.
• Pros: Ideal for high-level tools (like containerd or cri-o) that want full control over the container lifecycle without runc interference.
• Cons: More complex to implement. The parent process must call prctl(PR_SET_CHILD_SUBREAPER, 1, ...) to prevent zombie processes from accumulating on the host.

How to run in Detached Mode

You can trigger detached mode using either of these methods:

1. Use the -d flag with the run command: runc run -d ...
2. Follow the OCI lifecycle: runc create (to set up the container) followed by runc start (to begin execution).