ArcBox

repository·master·Indexed 41 days ago

https://github.com/arcboxlabs/arcbox

A high-performance, open-source container and VM runtime for macOS written in Rust. It serves as a drop-in replacement for Docker Desktop and OrbStack, supporting Docker containers, Kubernetes (via k3s), disposable microVM sandboxes, full Linux VMs, and macOS guests. It includes the `abctl` CLI for machine management and workload migration, and provides a Docker-compatible API server via the `arcbox-docker` crate.

Tokens
134.8K
Snippets
372
Records
609
Agent score
70%

What's inside ArcBox

  1. Overview of arcbox-fs

    master
    arcbox-fs is a high-performance filesystem service designed for ArcBox. It uses the VirtioFS protocol to facilitate file sharing between a host and a guest, providing near-native I/O performance. It works by intercepting FUSE protocol requests from the guest and mapping them directly to host filesystem operations.
  2. Overview of arcbox-docker features

    master

    The arcbox-docker crate provides a Docker-compatible API server that acts as a host-side proxy. It handles specific lifecycle orchestration via ArcBox handlers and forwards all other requests to the guest dockerd.

    Supported operations include:

    • Container Operations: create, start, stop, kill, rm, ps, inspect, logs, exec, attach, wait, pause, unpause, top, stats
    • Image Operations: pull, push, list, remove, tag, prune
    • Volume Operations: create, list, inspect, remove, prune
    • Network Operations: list, inspect, create, remove (basic)
    • System Operations: info, version, ping, events, df
  3. Overview of arcbox-virtio features

    master

    The arcbox-virtio crate provides paravirtualized device emulation for high-performance I/O between guest VMs and the host. Supported device types include:

    • virtio-blk: Block device with an async file backend.
    • virtio-net: Network device for VM networking.
    • virtio-console: Serial console I/O.
    • virtio-fs: Shared filesystem using the FUSE protocol.
    • virtio-vsock: Host-guest socket communication.
    • VirtQueue: Standard virtqueue implementation with descriptor chains.
  4. Overview of arcbox-api features

    master

    The arcbox-api crate provides gRPC service implementations built on top of the arcbox-core runtime state. It currently supports two primary service types:

    • MachineService: A gRPC service managing the machine lifecycle and facilitating guest-agent pass-through calls.
    • MigrationService: A gRPC service providing entrypoints for host-side migration planning and execution.
  5. Overview of arcbox-agent

    master

    The arcbox-agent is a guest-side agent designed to run inside ArcBox Linux guests. It facilitates communication between the host and the guest via vsock on port 1024.

    Its primary responsibilities include:

    • Host/Guest Liveness: Responding to Ping requests.
    • System Information: Providing guest system details.
    • Runtime Readiness: Ensuring the guest runtime stack (containerd, dockerd, or runc) is initialized and ready.
    • Runtime Status: Reporting the current state of the runtime.
    • System Initialization: When running as PID 1, it handles mounting filesystems, setting the hostname, and spawning a child reaper process.
    • Runtime Bootstrap: Automatically detects and launches the bundled runtime stack (containerd / dockerd / runc) to ensure the host-side Docker API proxy can connect to a healthy dockerd endpoint.
  6. Overview of arcbox-net networking modes and features

    master

    The arcbox-net crate provides a high-performance network stack for ArcBox VMs. It is designed with a zero-copy data path and a lock-free hot path using SPSC ring buffers to ensure minimal latency and maximum throughput.

    Networking Modes

    • NAT networking: Provides a default shared network with the host.
    • Bridge networking: Provides direct L2 connectivity.
    • Host-only networking: Creates isolated VM networks.

    Core Capabilities

    • Port forwarding: Expose guest services to the host.
    • DHCP server: Automatic IP assignment for guests.
    • DNS forwarding: Transparent DNS resolution.
    • Zero-copy packets: Direct guest memory access for high performance.
    • Lock-free data path: Uses SPSC ring buffers for the hot path.
  7. Overview of arcbox-cli

    master

    The arcbox-cli (commanded via abctl) is a thin command-line interface used for managing ArcBox machines and local integration helpers. It is important to note that arcbox-cli does not handle Runtime or Docker API serving itself; those tasks are managed by the separate arcbox-daemon binary.

    Key capabilities include:

    • Machine Management: Creating and managing Linux VMs.
    • Runtime Migration: Importing workloads from Docker Desktop or OrbStack.
    • Daemon Lifecycle: Starting and stopping the arcbox-daemon.
    • Docker Context Integration: Switching between standard Docker and ArcBox.
    • Kubernetes Integration: Managing the bundled k3s cluster and kubectl.
    • System Helpers: Managing boot assets and DNS resolver setup.
  8. Use arcbox-transport for host/guest communication

    master

    The arcbox-transport crate provides transport abstractions for communication between an ArcBox host and guest. It offers implementations for both Unix domain sockets and virtio-vsock endpoints, allowing for transport-agnostic code via the Transport and TransportListener traits.

    Available transport types:

    • UnixTransport: Uses Unix domain sockets.
    • VsockTransport: Uses virtio-vsock endpoints via VsockAddr.
    • VsockStream: Provides raw async byte streams for callers who already own a connected file descriptor (fd).
    use arcbox_transport::{Transport, UnixTransport, VsockTransport};
    use arcbox_transport::vsock::VsockAddr;
    use bytes::Bytes;
    
    // Example: Using Unix Domain Sockets
    let mut unix = UnixTransport::new("/var/run/arcbox.sock");
    unix.connect().await?;
    unix.send(Bytes::from("hello")).await?;
    
    // Example: Using Vsock
    let mut vsock = VsockTransport::new(VsockAddr::new(3, 1024));
    vsock.connect().await?;
    vsock.send(Bytes::from("ping")).await?;
  9. Current status of the ArcBox Python SDK

    master

    The Python SDK is currently in development (Phases 2a and 2b of CORE-58) and supports both synchronous and asynchronous surfaces. The following features are currently shipped:

    Sandbox Management

    • Sandbox.create, connect, list, kill, pause, info: Note that pause/resume are handled daemon-side; data-plane calls will automatically resume a paused sandbox.
    • sandbox.events(): Provides typed lifecycle events (sync reads are blocking; mid-stream drops raise ConnectionLostError).
    • sandbox.set_lifecycle(): Tri-state management (UNCHANGED to omit, None to restore default, or a specific value to replace).
    • sandbox.checkpoint(): Pauses and snapshots the sandbox, returning a Snapshot catalog row.
    • ArcBox.restore(snapshot_id): Creates a new READY sandbox from a snapshot. Supports fresh_network=True for concurrent restores. Includes list_snapshots (auto-paginating) and delete_snapshot.
    • arcbox.capabilities(): Performs a daemon handshake to retrieve version, protocol, features, and nested virtualization support (cached per client).

    Command Execution

    • commands.run: Executes commands. Returns a foreground result and a background CommandHandle. Supports stdin as str|bytes (write-then-close) or True (keep open), and pty=PtySize(cols, rows).
    • CommandHandle: Provides streamed output with transparent offset-resume (raises ConnectionLostError on persistent failure), wait_for_exit, kill, write_stdin, close_stdin, stdin_status, and resize.
    • handle.wait_for_log(pattern): Waits for the first log line matching a substring or compiled regex. Uses SDK-side replayed offset-addressed output. Raises TimeoutError on deadline expiration.
    • commands.get(id) / commands.list(): Re-attach a handle via execution ID or list running/exited executions as CommandInfo summaries.

    File System

    • files: Supports whole-file read/write and path verbs: stat, list, mkdir (with mkdir -p semantics), remove, and move. Uses a frozen FileStat dataclass. Paths accept str or PurePosixPath.
    • files.watch(path): Provides a typed FsEvent stream via AsyncFileWatch/FileWatch context managers. Renames are paired and keepalives are filtered. Mid-stream drops raise ConnectionLostError (events cannot be replayed).

    Networking

    • ports.wait_for_port(port): Waits for a guest-side listen-table entry (no client polling). Raises TimeoutError on expiry (daemon default 30s, cap 600s).
    • ports.expose(port) / unexpose / list: Publishes a guest port on the host loopback (TCP/UDP). list reads the daemon's authoritative live listener table.

    Templates

    • Template.build(name, ...): Builds a catalog template from a single source (image= local Docker ref, dockerfile= inline content, or snapshot= checkpoint promotion). Supports ready_probe_port, ready_probe_command, and prewarm=True (boot-to-READY warm snapshot). Builds block until complete.
    • Template.get, list, delete_reference: Resolves name[:version] (bare name is newest published; otherwise draft). Supports auto-paginating lists.
    • instance.publish(version) / delete(): publish freezes a draft immutably; delete removes it.
    • create(template=Template | "name[:version]"): Creates a sandbox. A Template instance pins its exact reference. Overriding vcpus or memory_mib replaces limits wholesale; no_default_cmd or no_default_env suppresses inherited values.
  10. Use arcbox-container for container models and exec orchestration

    master

    The arcbox-container crate provides the foundational data models and execution primitives for managing containers within the ArcBox ecosystem. It is designed to be used by higher-level services rather than acting as a standalone container manager.

    Key components include:

    • Container Configuration: Use ContainerConfig to define parameters for container creation.
    • Container Identity & Lifecycle: Use Container and ContainerState to track container identity and its current lifecycle status.
    • Execution Orchestration: Use ExecManager and related types (like ExecInstance, ExecConfig, and ExecId) to implement docker exec-style workflows.
    • Agent-backed Execution: Supports optional execution operations via ExecAgentConnection.
    use arcbox_container::{Container, ContainerConfig, ContainerState};
    
    let config = ContainerConfig {
        image: "alpine:latest".to_string(),
        cmd: vec!["sh".to_string()],
        ..Default::default()
    };
    
    let container = Container::with_config("demo", config);
    assert_eq!(container.state, ContainerState::Created);
  11. Use arcbox-grpc gRPC bindings

    master

    The arcbox-grpc crate provides gRPC client and server bindings for ArcBox services using tonic. It contains service modules generated from the arcbox.v1 package, with protobuf messages sourced from arcbox-protocol.

    All generated services are accessible under the arcbox_grpc::v1::* namespace. For convenience, the crate root re-exports several primary clients and servers.

    use arcbox_grpc::MachineServiceClient;
    use arcbox_protocol::v1::ListMachinesRequest;
    
    let request = tonic::Request::new(ListMachinesRequest { all: true });
    // client.list(request).await?;
  12. What is splicetcp and how does it work

    master

    splicetcp is a userspace TCP-termination and splice bridge used in ArcBox. It acts as a frame plane between an L2/L3 packet source and an egress dialer.

    Key Characteristics:

    • Not a full TCP/IP stack: The guest kernel maintains ownership of the guest TCP state. splicetcp only manages outbound SYNs, synthesizes the handshake, mirrors minimal sequence/ack state, and splices bytes between guest frames and host sockets.
    • Flow Promotion: It classifies Ethernet frames and promotes established TCP flows to a "fast path" backed by a real host TcpStream.
    • Core Components:
      • classifier::FrameClassifier: A pure Ethernet-frame demultiplexer.
      • tcp_bridge::TcpBridge: Handles SYN gating, handshake synthesis, and the fast path.
      • egress::EgressResolver: The policy/dial seam used to map flows to outbound connections.
      • direct_rx::{ConnSink, PromotedConn}: Allows callers to take ownership of host→guest reads.
      • shim: Adapts L3 links (like utun) to the L2 classifier by adding/removing synthetic Ethernet headers.