DOCSEARCHbyAlgolia
Submit Docs

vNode Documentation

website·Indexed 32 days ago

https://vnode.com/docs/

vNode is a tenant isolation container runtime for Kubernetes that provides strong node-level isolation between workloads using Linux user namespaces. It integrates with containerd as a RuntimeClass to deliver a lightweight sandbox for pods, offering per-namespace or per-pod isolation granularity and compatibility with vCluster for combined control-plane and node-level isolation.

Tokens
8.5K
Snippets
32
Records
57
Agent score
65%

What's inside vNode

  1. Overview of vNode for Kubernetes Multi-Tenancy

    vNode provides node-level isolation by creating virtual nodes within a shared physical Kubernetes node. This allows for stronger tenant isolation, preventing 'noisy neighbor' issues and cross-tenant interference, while maintaining the cost and performance benefits of shared infrastructure. It is designed to support containerd-based Kubernetes environments on Linux nodes (v1.6+) across public and private clouds, with full CPU and GPU support.

  2. Understand vNode isolation mechanisms

    vNode provides a lightweight sandbox for Kubernetes pods that acts as a minimal host environment. It achieves isolation without a hypervisor or guest kernel by using three kernel-level mechanisms: Linux user namespaces, FUSE-based filesystem virtualization, and targeted seccomp filtering. This ensures that processes inside a vNode cannot access other tenants' files, processes, or hardware information, even in the event of a container escape.

  3. vNode Architecture Components Reference

    The vNode runtime is composed of several specialized components:

    • vNode Manager (vnode-manager): A multi-call binary acting as the CNI plugin (vnode-cni), the containerd shim plugin (containerd-shim-vnode-v2), and the manager service providing FUSE-based /proc and /sys filesystems and seccomp filtering.
    • vNode Init (vnode-init): Runs as PID 1 inside each vNode to reap zombie containerd-shim processes.
    • vNode RunC (vnode-runc): A fork of RunC that integrates the vNode manager and supports cgroup v2 and Linux user namespaces.
    • vNode Containerd Shim (vnode-containerd-shim-runc-v2): Handles ID-mapped mounts and sets up FUSE, seccomp, and the manager API for new vNodes.
    • vNode Utils (vnode-utils): Performs helper operations like network namespace setup, bind-path operations, and file copying into vNodes.

  4. Implement full-stack tenant isolation with vCluster and vNode

    Full-stack tenant isolation is achieved by combining two layers: vCluster for control-plane isolation and vNode for node-level isolation. vCluster provides each tenant with a dedicated API server and etcd, preventing tenants from interfering with each other's secrets or exhausting API capacity. vNode wraps tenant pods in a hardened sandbox, ensuring that a container escape only reaches the vNode boundary rather than the host node, protecting other tenants' filesystems and processes.

  5. Understand the vNode escape landing zone

    The 'escape landing zone' refers to the state of an attacker after they have successfully broken out of a container. In vNode, the isolation is enforced at the kernel level rather than via policy or admission controls. Key guarantees after an escape include:

    • No root access: The workload's root process (UID 0) maps to a high-numbered unprivileged host UID (65536 or higher).
    • Tenant Isolation: Each vNode uses a distinct UID range; kernel file permission checks prevent access to other tenants' files.
    • Process Isolation: PID namespaces ensure that only the attacker's own processes are visible in /proc.
    • Network Restriction: Seccomp unconditionally blocks AF_PACKET and SIOCSIFFLAGS (promiscuous mode), preventing raw network access.

  6. Understand vNode's three-layer isolation architecture

    vNode secures workloads using three layered kernel-level mechanisms to ensure that even in the event of a container escape, the attacker remains an unprivileged user on the host (UID 65536 or higher) with no access to other tenants' processes, files, or raw network access.

    1. Layer 1: User Namespaces and UID Mapping: Each vNode is assigned a unique range of 65,536 IDs. Container root (UID 0) is mapped to an unprivileged host UID (≥ 65,536). ID-mapped mounts handle this translation without performance penalties.
    2. Layer 2: FUSE Filesystem Virtualization: vNode uses a FUSE filesystem (vnodefs) to mask sensitive host-level information in /proc and /sys. It returns per-container data (e.g., container-specific uptime in /proc/uptime) and hides hardware identifiers in /sys/devices/virtual/dmi.
    3. Layer 3: Targeted Seccomp Filtering: vNode uses surgical seccomp rules rather than full kernel reimplementation:
      • Blocked: Raw packet socket operations (AF_PACKET, promiscuous mode, packet multicast) are hard-denied to prevent cross-tenant traffic sniffing.
      • Intercepted: Mount operations, ownership changes, specific xattr operations, and reboot are handled in userspace via seccomp-notify by the vnode-manager.
      • Pass-through: All other syscalls hit the real kernel, relying on standard permission checks for the unprivileged host UID.

  7. Install vNode using Helm

    Install the vNode runtime using Helm. You must provide the platform host URL and the platform access key. If your platform uses a self-signed certificate, add the config.platform.insecure=true flag.
    helm upgrade --install vnode-runtime vnode-runtime -n vnode-runtime \
--repo https://charts.loft.sh --create-namespace \
--set "config.platform.host=https://platform-your-domain.com" \
--set "config.platform.accessKey=your-access-key"

# Add the following only if your platform uses a self-signed certificate:
# --set "config.platform.insecure=true"

  8. Secure untrusted workloads for Inference Providers

    Inference providers should use vNode to sandbox untrusted or privileged customer code to prevent container escapes from affecting the host node. This is critical for:

    • Custom or customer-supplied serving containers.
    • Model adapters, plugins, or extensions loaded at runtime.
    • Fine-tuning or training jobs running alongside serving.
    • Privileged GPU workloads requiring device access beyond namespace boundaries.

    Recommended Architecture: Layer vNode on top of Private GPU nodes. Private nodes provide the necessary infrastructure boundary, and vNode provides the runtime sandbox to contain escapes.

  9. Verify vNode installation and isolation

    To verify that vNode is correctly installed and providing isolation, create a privileged test pod using the vnode runtime class and check for UID remapping and process isolation.
    # 1. Create a privileged test pod
echo "apiVersion: v1
kind: Pod
metadata:
  name: vnode-test
spec:
  runtimeClassName: vnode
  hostPID: true
  terminationGracePeriodSeconds: 1
  containers:
    - image: ubuntu:jammy
      name: vnode-test
      command: ['tail', '-f', '/dev/null']
      securityContext:
        privileged: true" | kubectl apply -f -

kubectl wait --for=condition=ready pod vnode-test

# 2. Verify UID remapping
# Root should be remapped to a high unprivileged host UID (e.g., 589824)
kubectl exec vnode-test -- cat /proc/self/uid_map

# 3. Verify process isolation
# Should only show pod-specific processes, not host processes
kubectl exec -it vnode-test -- ps -ef --forest

  10. Install vNode using Helm

    Install the vNode runtime using the Helm chart from the Loft repository. You must provide the platform host and access key. If your platform uses a self-signed certificate, add the config.platform.insecure=true flag.
    helm upgrade --install vnode-runtime vnode-runtime -n vnode-runtime \
--repo https://charts.loft.sh --create-namespace \
--set "config.platform.host=https://platform-your-domain.com" \
--set "config.platform.accessKey=your-access-key"

# Add the following only if your platform uses a self-signed certificate:
# --set "config.platform.insecure=true"

  11. Prerequisites for installing vNode

    Before installing vNode, ensure the following requirements are met:

    • An existing vCluster Platform installation.
    • A license plan that includes vNode (verify via Platform Config → License and Billing).
    • A valid platform access key (not required when deploying inside a Tenant Cluster).
    • A Kubernetes cluster with Linux kernel 6.1+, containerd 1.7 or 2.0, and Kubernetes 1.24+.

  12. Determine when to use vNode for workload isolation

    vNode provides strong workload isolation by wrapping workloads in hardened sandboxes that contain container escapes at the vNode boundary. It is the only model that adds strong isolation while keeping tenants on shared worker nodes.

    Use vNode when:

    • Isolation must exceed namespace boundaries, but dedicated nodes are too expensive.
    • Tenants run privileged workloads (e.g., Docker-in-Docker, nested Kubernetes, or GPU workloads) on shared infrastructure.
    • Compliance requirements prohibit namespace-only isolation but do not mandate fully dedicated nodes.

    Note: vNode is not a replacement for dedicated nodes if the tenant requires fully separate infrastructure, dedicated CNI/CSI, or independent node lifecycle control.