DOCSEARCHbyAlgolia
Submit Docs

E2B Infrastructure

repository·main·Indexed 22 days ago

https://github.com/e2b-dev/infra

Backend foundation for AI code interpreting, providing components to power the E2B platform. Includes documentation for Terraform deployments on major cloud providers, the envd sandbox daemon, Redis Reservation Storage for sandbox management, Nomad Node Pool Autoscaler plugins (nomad-nodepool-apm and nomad-deployment-aware-target), and the Orchestrator for managing sandbox builds and snapshots.

Tokens
23.6K
Snippets
29
Records
113
Agent score
79%

What's inside e2b-dev-infra

  1. Overview of E2B Infrastructure

    main
    E2B Infrastructure is the open-source foundation that powers the E2B platform, specifically designed for AI code interpreting. While the main e2b-dev/e2b repository provides the SDKs and CLI for managing environments and running AI agents, this repository contains the underlying infrastructure components.

  2. Overview of E2B Infrastructure services

    main

    The E2B infrastructure is composed of several specialized services organized into functional layers:

    Control Plane

    • API (packages/api): The central REST and gRPC entry point for managing sandboxes, templates, and builds. It uses PostgreSQL for persistent state and Redis for real-time tracking.
    • Dashboard API (packages/dashboard-api): Serves the dashboard interface, interacting with PostgreSQL and ClickHouse.
    • Client Proxy (packages/client-proxy): Handles routing for sandbox traffic (e.g., port-sandboxid.domain) by looking up sandbox locations in Redis and forwarding requests to the appropriate orchestrator.
    • Docker Reverse Proxy (packages/docker-reverse-proxy): Manages Docker-related traffic.

    Data Plane & Execution

    • Orchestrator (packages/orchestrator): Runs on sandbox nodes. It manages Firecracker microVMs, networking, and storage. It communicates with the API via gRPC.
    • Envd (packages/envd): An in-VM agent running inside the Firecracker microVM to manage user processes.
    • Template Manager (packages/template-manager): A specialized service (using the orchestrator binary) that handles template creation and interacts with object storage.

    Data Stores

    • PostgreSQL: Stores teams, templates, builds, and snapshots.
    • Redis: Tracks running sandboxes, the routing catalog, and caches.
    • ClickHouse: Stores metrics, events, and optional logs.
    • Object Storage (GCS/S3): Stores template and snapshot artifacts.

  3. Understand the E2B Infrastructure service layout

    main

    The E2B infrastructure is composed of several specialized packages that manage the lifecycle of sandboxes, traffic routing, and data storage:

    • api: The control-plane REST API.
    • orchestrator: The sandbox runtime and template builder. It runs as a single binary per node.
    • client-proxy: An edge router used for sandbox traffic.
    • envd: An agent that runs inside the VM.
    • dashboard-api: The backend for the web dashboard.
    • docker-reverse-proxy: A registry authentication gateway for template images.
    • auth: An authentication library (supporting API keys, JWT/OIDC) used by the api and dashboard-api.
    • db: Contains Postgres migrations (using goose) and queries (using sqlc).
    • clickhouse: Manages ClickHouse schemas, batching writers, and query clients.
    • shared: Contains common protos, telemetry, storage clients, the proxy engine, and feature flags.
    • nomad-nodepool-apm: Provides Nomad autoscaler metric and deployment-aware target plugins.
    • local-dev: Provides a docker-compose local stack and DB seeding for development.

  5. Overview of E2B Data Stores

    main

    E2B uses several specialized data stores to manage different aspects of the infrastructure:

    • PostgreSQL (packages/db): Stores durable control-plane state including teams, users, tiers, project_limits, envs (templates), env_builds, env_aliases, snapshots (paused sandboxes), team_api_keys, access_tokens, volumes, and clusters.
    • Redis: Manages ephemeral runtime state, acting as the source of truth for running sandboxes, the sandbox→node routing catalog, caches (team/template/snapshot), rate limiting, and the P2P chunk peer registry.
    • ClickHouse (packages/clickhouse): Handles time-series and analytics data such as metrics_gauge/metrics_sum (via OTel), sandbox_events, sandbox_host_stats, and team metrics.
    • Object Storage (GCS/S3/local, packages/shared/pkg/storage): Stores template and snapshot artifacts. Artifacts are keyed by buildID and include {buildID}/memfile, {buildID}/rootfs.ext4, {buildID}/snapfile, and {buildID}/metadata.json + .header index files.
    • Consul KV: Used by the orchestrator for network slot allocation across restarts.

  6. How nomad-deployment-aware-target works

    main

    The nomad-deployment-aware-target manages scaling to prevent conflicts with active Nomad deployments:

    1. It serializes scaling per namespaced job and acts as a no-op if the current count already matches the desired count.
    2. If the count must change, it intentionally fails any conflicting active deployment (Nomad will record the deployment as failed rather than cancelled).
    3. It rereads the job and scales using the current job modify index.
    4. It verifies the final count; the new rollout proceeds normally.
    5. Concurrent changes are retried with a bounded attempt count.

    Note: Dry-run actions do not fail deployments or write task-group counts.

  7. How the Client Proxy handles sandbox traffic

    main

    The Client Proxy (packages/client-proxy) is a stateless edge router (port 3002) that terminates requests for https://<port>-<sandboxID>.<domain>.

    Traffic Flow:

    1. The proxy parses the host to identify the sandboxID.
    2. It looks up the sandbox in the Redis routing catalog to find the owning node.
    3. It reverse-proxies the request to that node's orchestrator proxy on port 5007.
    4. Transparent Wake: If the sandbox is not in the catalog (meaning it is paused), the proxy calls the API's ResumeSandbox gRPC method and retries the request, allowing paused sandboxes to wake up transparently upon incoming traffic.

  8. AWS Architecture for E2B deployment

    main

    The AWS deployment uses EC2 Auto Scaling Groups for various node pools:

    • Control Server: Nomad/Consul servers (default: 3x t3.medium).
    • API: API server, ingress, client proxy, otel, loki, logs collector (default: t3.xlarge).
    • Client: Firecracker orchestrator nodes with nested virtualization (default: m8i.4xlarge).
    • Build: Template manager for building sandbox templates (default: m8i.2xlarge).
    • ClickHouse: Analytics database (default: t3.xlarge).

    Managed Services:

    • ElastiCache Redis: Can be enabled by setting REDIS_MANAGED=true.

  9. Understand the Redis Reservation Storage flow

    main

    The Redis Reservation Storage package manages sandbox creation reservations across multiple API instances to prevent over-provisioning and ensure atomicity.

    The Reservation Lifecycle

    1. Reserve: Executes a Lua script that performs several atomic operations:

      • Removes stale pending entries.
      • Checks if the sandbox already exists or has a pending start.
      • Enforces team limits by calculating SCARD(storage index) + ZCARD(pending zset).
      • Deletes stale result keys.
      • Adds the sandboxID to the pending zset.

    2. Completion: Once creation is finished, the system:

      • Removes the sandbox from the pending zset.
      • Writes a result key with a TTL containing either the sandbox object or a creation error.
      • Publishes a message to the PubSub routing key.

    3. Waiting: A waiter process subscribes to the PubSub routing key, probes the result key immediately, and then waits for either a PubSub notification or a 1-second fallback ticker. Note: PubSub is best-effort; the 1-second fallback ticker is required to ensure correctness.

    4. Release: Triggered when a sandbox is removed from storage (Store.Remove). It removes the sandbox from the pending zset, deletes the result key, and publishes to the routing key.

  10. How nomad-nodepool-apm works

    main

    The nomad-nodepool-apm plugin follows these steps to provide metrics to the Nomad Autoscaler:

    1. Queries the Nomad API to list nodes filtered by the node pool name provided in the query parameter.
    2. Counts only nodes with a status of ready.
    3. Returns this count as a metric.
    4. When used with a pass-through strategy, this count is treated as the desired number of allocations.

  11. Configure architecture and cross-architecture deployment

    main

    The orchestrator supports amd64 and arm64. While architecture is detected automatically at compile time, you can override runtime behavior using environment variables.

    Cross-Architecture Deployment

    Use the TARGET_ARCH environment variable to override the architecture used for path resolution (kernels/Firecracker) and OCI image pulls. This is a runtime setting.

    # Run orchestrator targeting amd64 paths from an arm64 host
TARGET_ARCH=amd64 ./bin/orchestrator

    Building for ARM64

    To build the binaries themselves for ARM64, use the BUILD_ARCH environment variable during the build process.

    # Build ARM64 binaries
BUILD_ARCH=arm64 make build-local

    Architecture Naming Convention

    The project uses Go/Docker/Debian naming (amd64/arm64) for directory paths and GCS buckets, rather than Linux/GNU naming (x86_64/aarch64).

    TARGET_ARCH=amd64 ./bin/orchestrator

  12. Kernel and Module Limitations in Sandboxes

    main

    Sandboxes always boot using the kernel supplied by E2B, rather than the kernel from the base image. This leads to the following constraints:

    • /lib/modules is always empty.
    • No kernel modules can be loaded.
    • SELinux is disabled.
    • Installing kernel-devel from a distro's repositories will resolve against a kernel that is not actually running.

    Important Compatibility Note: Because of these kernel constraints, specific distributions that rely on specific kABI, signed kernel modules, or UEK are not accepted, even if they use supported package managers (RPM/dnf). These include:

    • RHEL (including UBI)
    • Oracle Linux (ol)
    • Amazon Linux (amzn)

    Community rebuilds of these distributions are supported only if they are chosen for their userland rather than their kernel properties.