DOCSEARCHbyAlgolia
Submit Docs

CubeSandbox Documentation

repository·master·Indexed 27 days ago

https://github.com/tencentcloud/cubesandbox

A secure, concurrent sandbox service optimized for AI Agents, providing hardware-level isolation with low overhead. The project includes cube-agent (an in-VM guest agent for container lifecycle management), cube-lifecycle-manager (a service for sandbox auto-pause and auto-resume), CubeAPI (an E2B-compatible API server), and cubecow (a Copy-On-Write storage engine utilizing xfs-reflink for O(1) snapshots and clones).

Tokens
221.8K
Snippets
484
Records
1.2K
Agent score
94%

What's inside CubeSandbox

  1. Overview of CubeSandbox capabilities

    master

    CubeSandbox is a high-performance, secure, and lightweight sandbox service designed for AI Agents, built on RustVMM and KVM. It provides hardware-level isolation with MicroVMs and is compatible with the E2B SDK, allowing for seamless transitions from E2B cloud to self-hosted environments via environment variable changes.

    Key features include:

    • Ultra-fast startup: Average cold start < 60ms using resource pooling and snapshot cloning.
    • Hardware-level isolation: Each sandbox runs its own OS kernel in a dedicated MicroVM.
    • High density: Memory overhead per sandbox is < 5MB, supporting thousands of instances per machine via kernel sharing and Copy-on-Write (CoW).
    • E2B Compatibility: Compatible with E2B SDK interfaces.
    • Network Security: eBPF-based kernel-level isolation and outbound filtering, plus an L7 security proxy for fine-grained policy control and credential injection.
    • State Management: High-frequency snapshots and rollbacks (via CubeCoW) with millisecond-level performance.
    • Volume Framework: Supports custom backend storage via plugins, compatible with E2B standards.
    • Deployment Options: Supports single-machine, multi-machine clusters, Tencent Cloud (via Terraform), and Kubernetes (preview).

  2. Overview of cubecow storage engine

    master

    cubecow is a Copy-On-Write (CoW) storage engine implemented as a Rust library. It provides thin-provisioned volume management and O(1) snapshot/clone capabilities using the xfs-reflink (ReflinkEngine) backend. It leverages the kernel FICLONE ioctl to perform metadata-only snapshots, meaning extents are shared rather than byte-copied.

    Key characteristics:

    • O(1) snapshots and clones: Cost depends only on the source file's extent count.
    • Flat snapshot model: Every snapshot is an independent file; deleting one does not affect others.
    • Simple crash recovery: The in-memory index is rebuilt by scanning the <root_dir>/volumes/ directory on restart.
    • Backend-agnostic API: Uses the Engine trait to allow for future backend implementations without changing FFI/SDK callers.

  4. Overview of Cube Sandbox core components

    master

    Cube Sandbox architecture is divided into a stateless Control Plane and a node-local Data Plane:

    Control Plane (Stateless)

    • CubeAPI: Rust-based (Axum) E2B-compatible REST gateway. Translates SDK calls to gRPC.
    • CubeMaster: Go-based cluster orchestrator. Handles scheduling and dispatches work to nodes.
    • Redis: The single source of truth for sandbox metadata, lifecycle events, and routing tables.

    Data Plane (Node-local)

    • Cubelet: Go-based agent managing the full sandbox lifecycle on a specific node.
    • CubeShim: Rust-based containerd Shim v2 implementation that bridges the runtime to the MicroVM.
    • CubeHypervisor: RustVMM + KVM-based VMM managing vCPU, memory, and virtio devices.
    • CubeCoW: Rust library for O(1) storage snapshots and clones using XFS reflink (FICLONE).
    • CubeVS: eBPF-based network data plane for SNAT/DNAT and policy enforcement.
    • CubeEgress: L7 proxy for outbound traffic security.

  5. Overview of cube-lifecycle-manager

    master

    The cube-lifecycle-manager is a standalone service responsible for coordinating sandbox auto-pause and auto-resume operations between CubeMaster, CubeProxy, and Redis.

    Key responsibilities include:

    • Consuming lifecycle events from the cube:v1:shared:sandbox:lifecycle:events stream.
    • Real-time discovery of live CubeProxy replicas via cube:v1:shared:cube_proxy:{registry,heartbeat}.
    • Broadcasting sandbox metadata and state through CubeProxy /admin/* endpoints.
    • Handling synchronous /internal/resume callbacks invoked by CubeProxy when a paused sandbox receives a request.

  6. Overview of Cube Sandbox One-Click

    master

    The deploy/one-click directory is used to build and deliver a single-machine, one-click release package for cube-sandbox. This package includes all necessary components, guest images, kernels, and configuration templates for a standalone deployment.

    Key Components in the Directory:

    • build-release-bundle-builder.sh: The recommended entry point for building the release bundle.
    • install.sh: Entry point for installing and starting the control node (all-in-one mode) on the target machine.
    • install-compute.sh: Entry point for installing compute nodes on the target machine.
    • down.sh: Stops the services and dependencies installed by the one-click package.
    • smoke.sh: Executes basic health checks.
    • config-cube.toml: Default runtime configuration template for the one-click deployment.
    • terraform/tencentcloud/: Terraform deployers for deploying the cluster version of CubeSandbox on Tencent Cloud (TKE control plane + CVM compute nodes).

  7. Overview of CubeVS Network Architecture

    master

    CubeVS is a network virtualization layer that isolates sandboxes using eBPF programs and shared BPF maps. It replaces traditional Linux Bridge or OVS stacks with three specific eBPF programs attached to network boundaries to minimize per-packet overhead and enforce security policies in kernel space.

    Key design features:

    • No shared bridge: Each sandbox uses its own TAP device for direct kernel data path attachment.
    • Kernel-space enforcement: Policies are evaluated in eBPF before reaching userspace.
    • Scalable NAT: Uses a lock-protected pool for SNAT port allocation to avoid iptables rule explosion.

  8. Overview of Cloud Hypervisor API interfaces

    master

    The Cloud Hypervisor API consists of two distinct interfaces:

    1. External API: The user-facing interface used by operators to control and manage Cloud Hypervisor. It is accessible via a REST API or a Command Line Interface (CLI).
    2. Internal API: An interface used for communication between Cloud Hypervisor threads, based on Rust's Multi-Producer, Single-Consumer (MPSC) module.

  9. Overview of CubeShim

    master

    CubeShim is a containerd shim component implemented in Rust. It implements the containerd Shim v2 API and acts as a bridge between containerd and the Cube sandbox VM lifecycle.

    Key Responsibilities:

    • VM Creation: Communicates with Cubelet to launch KVM MicroVMs with specified CPU, memory, and disk resources.
    • Lifecycle Management: Forwards Create, Start, Exec, Kill, and Delete calls to the in-VM cube-agent via ttrpc/vsock.
    • I/O & Signaling: Proxies stdio streams and forwards signals between the host and the container process inside the VM.
    • State Reporting: Tracks sandbox status and surfaces it to containerd via the Shim v2 event model.

  10. Overview of cube-agent

    master

    The cube-agent is the in-VM guest agent for Cube Sandbox. It runs as PID 1 (init) inside each MicroVM and is responsible for managing the full container lifecycle within the sandbox.

    Key responsibilities include:

    • Initializing the guest environment: Mounting filesystems, setting up namespaces, and configuring networking via vsock/netlink.
    • Listening for ttrpc commands: Exposing the Cube agent API over a vsock channel for the shim (containerd-shim-cube-rs) to drive.
    • Managing container lifecycle: Handling CreateContainer, StartContainer, ExecProcess, SignalProcess, and RemoveContainer requests using rustjail for OCI-compliant execution.
    • I/O Forwarding: Proxying container stdio streams back to the shim over vsock.
    • Metrics: Exporting Prometheus-compatible metrics for guest CPU, memory, and container health.

  12. Overview of CubeSandbox

    master

    CubeSandbox is a purpose-built infrastructure for AI Agents, providing a production-hardened runtime service for hosting Agent processes and stateful services. It uses hardware-isolated MicroVMs built on RustVMM and KVM to ensure secure execution of untrusted LLM-generated code.

    Key technical characteristics include:

    • High Performance: Sub-60ms boot times and <5MB memory overhead per instance.
    • Security: Hardware-level isolation with dedicated Guest OS kernels and eBPF-enforced boundaries.
    • Agent-Centric Features: Supports long-running processes, auto-pause/auto-resume (snapshot-suspended), and sub-second snapshot/clone/rollback capabilities via CubeCoW.
    • Compatibility: Native E2B SDK compatibility, allowing for seamless migration by swapping a single URL environment variable.
    • Egress Control: Managed via CubeEgress, providing L7 domain filtering and auditing.