Kaniko

kaniko is a tool designed to build container images from a Dockerfile inside a container or a Kubernetes cluster.

Unlike traditional tools, kaniko does not depend on a Docker daemon. Instead, it executes each command within a Dockerfile completely in userspace. This makes it suitable for environments that cannot easily or securely run a Docker daemon, such as standard Kubernetes clusters.

How it works:

1. The kaniko executor image extracts the filesystem of the base image (the FROM image).
2. It executes the commands specified in the Dockerfile.
3. After each command, it snapshots the filesystem in userspace.
4. It appends a layer of changed files to the base image and updates the image metadata.

Kaniko builds container images by executing Dockerfile commands in user-space without requiring a privileged Docker daemon. The process follows these steps:

1. Initialization: The builder executable extracts the filesystem of the base image (defined in the FROM line) to root.
2. Execution: It parses the Dockerfile and executes each command in sequence.
3. Snapshotting: After each command, Kaniko snapshots the filesystem by comparing checksums of files before and after execution. Changes are stored as tarballs.
4. Layering: These tarball snapshots are treated as new layers.
5. Pushing: Once all commands are complete, Kaniko appends the new layers to the base image and pushes the final image to the specified registry destination.

Kaniko is designed as an alternative to mounting the host Docker socket into a Pod. Mounting the Docker socket (/var/run/docker.sock) via hostPath gives the container full root access to the host machine, allowing it to create, delete, and modify other containers on the same node.

Security and Portability Risks:

• Security: It provides full root access to the host.
• Leaky Abstraction: Containers become dependent on the specific container runtime of the node, preventing them from running on nodes using alternate Container Runtime Interfaces (CRI) or virtual kubelets.

apiVersion: v1
kind: Pod
metadata:
  name: docker
spec:
  containers:
  - image: docker
    command: ["docker", "ps"]
    name: docker
    volumeMounts:
    - name: dockersock
      mountPath: /var/run/
  volumes:
  - name: dockersock
    hostPath:
      path: /var/run/

Kaniko can cache layers in a remote repository to speed up builds. Before executing a command, kaniko checks the cache; if found, it pulls and extracts the layer instead of executing the command. If not found, it executes the command and pushes the new layer to the cache.

• Use --cache-run-layers to cache layers created by RUN commands.
• Use --cache-copy-layers to cache layers created by COPY commands.

Docker-in-Docker (DinD) runs a second Docker daemon inside a container, which itself runs inside the node's Docker daemon.

Drawbacks:

• Complexity: Docker and cgroups do not handle nesting well, which can cause bugs and unexpected behavior.
• Privilege Requirements: This approach requires the --privileged flag on the outer container. In Kubernetes, this must be explicitly allowed by the node's configuration before the pod can be scheduled.

Kaniko uses file mtime (modification time) in its hashing algorithms to determine if a file has changed during snapshotting. Because there can be a delay between a file change and its mtime being updated, users should be aware of the following behaviors:

• --snapshot-mode=time: Kaniko may miss changes introduced by RUN commands entirely if the mtime has not updated.
• --snapshot-mode=full (Default): If a RUN command modifies a file's metadata but not its actual contents, whether or not kaniko adds a new layer is theoretically non-deterministic. This does not affect the correctness of the image contents, but it can affect the total number of layers produced.

Kaniko requires a build context (the files and Dockerfile needed for the build). In this tutorial, you create a local directory on the Kubernetes node and mount it using a hostPath volume.

1. SSH into your cluster (e.g., using minikube ssh).
2. Create a directory and a simple dockerfile inside it.
3. Important: Note the absolute path of this directory; you must use this path in the hostPath field of your volume.yaml configuration.

$ mkdir kaniko && cd kaniko
$ echo 'FROM ubuntu' >> dockerfile
$ echo 'ENTRYPOINT ["/bin/bash", "-c", "echo hello"]' >> dockerfile
$ cat dockerfile
FROM ubuntu
ENTRYPOINT ["/bin/bash", "-c", "echo hello"]
$ pwd
/home/<user-name>/kaniko # copy this path in volume.yaml file

kaniko is intended to be run as a complete image. The recommended image to use is gcr.io/kaniko-project/executor.

Warning: Do not attempt to run the kaniko executor binary inside another image (for example, by copying it into a Jenkins CI agent). Doing so is not supported and may not work as expected because kaniko unpacks directly into its own container root and may overwrite existing files, as it cannot use chroot or bind-mounts without requiring elevated privileges.