DOCSEARCHbyAlgolia
Submit Docs

Ethereum Package

repository·main·Indexed 19 days ago

https://github.com/ethpandaops/ethereum-package

A Kurtosis-based tool for deploying reproducible, multi-client Ethereum testnets (private or shadowforked) on Docker or Kubernetes. It supports various Execution Layer (EL) and Consensus Layer (CL) clients, PBS emulation, and observability tools like Grafana and Prometheus. Key features include shadowforking public networks, network condition simulation via Disruptoor, and the ability to inject local binaries for rapid debugging.

Tokens
15.9K
Snippets
53
Records
69
Agent score
17%

What's inside ethereum-package

  1. Overview of Ethereum Package features

    main

    The Ethereum Package is a Kurtosis package used to spin up private Ethereum testnets or shadowfork public networks.

    Core capabilities include:

    • Generating EL/CL genesis data.
    • Bootstrapping a configurable number of nodes.
    • Running a transaction spammer (spamoor).
    • Providing observability via Grafana and Prometheus.
    • Analyzing blob transactions via Blobscan.

    Optional features (via flags/params):

    • PBS Emulation: Support for flashbots, helix, mev-rs, commit-boost, or mock relays.
    • JSON RPC Snooper: Logs requests/responses between EL engine API and CL client.
    • TrueBlocks: Serves the chifra REST API on port 8080 for indexing.
    • OTel Tracing: Ships traces to a ClickHouse instance (requires otel in additional_services).

  2. How the Participant Network is constructed

    main

    The participant network is the core component responsible for managing EL and CL clients. It operates in two main phases:

    1. Data Generation

    All EL clients require a genesis file and a JWT secret. The package uses a specialized Docker image to generate these files by filling in a template found in static_files/genesis-generation-config/el-cl/values.env.tmpl.

    CL clients also receive genesis and config files generated during this phase. Additionally, validator keys are generated using eth2-val-tools and stored as file artifacts.

    2. Client Launching

    • EL Clients: Launched using EL client 'launchers' (e.g., Geth, Besu). The launch function consumes EL genesis data and returns an el_context struct containing node information.
    • CL Clients: Launched using CL client launchers (e.g., Nimbus, Lighthouse). The cl_client_launcher.launch method requires both the CL genesis data/keys and an el_context, reflecting the 1:1 relationship between CL and EL clients. Launched CL nodes are tracked in a cl_context struct.

  3. Configure the package via Package I/O

    main

    Configuration options are passed to the package's main function as a YAML or JSON-serialized string via the input_args variable.

    To modify how the package behaves, you can inspect the src/package_io directory to:

    • View or change available parameters.
    • Change default parameter values.
    • View or change the validation logic applied to configuration.
    • View or change the properties returned to the user after execution.

  4. How the Ethereum Package operates

    main

    The Ethereum Package is a Kurtosis package that automates the deployment of an Ethereum network. The operational lifecycle follows these steps:

    1. Parameter Parsing: The package parses user-provided configuration.
    2. Participant Network Launch:
      • Generates Execution Layer (EL) client configuration data.
      • Launches EL clients.
      • Generates Consensus Layer (CL) client configuration data.
      • Launches CL clients.
    3. Auxiliary Services: Launches supporting services like Grafana, Prometheus, and Forkmon.
    4. Merge Verification: Runs logic to ensure the Ethereum Merge has occurred successfully.
    5. Response: Returns network information to the user.

  5. Emulate Proposer Builder Separation (PBS) with different MEV types

    main

    You can emulate different MEV (Maximal Extractable Value) environments by setting the mev_type configuration parameter.

    Supported mev_type values:

    • flashbots: Instantiates a full Flashbots-style infrastructure including a modified Geth builder, mev-relay-api, mev-relay-website, mev-relay-housekeeper, and mev-boost for each EL/CL pair.
    • helix: Uses the high-performance Helix relay with a TimescaleDB backend.
    • mev-rs: Uses the mev-rs relay implementation.
    • commit-boost: Uses infrastructure powered by commit-boost.
    • buildoor: Uses a self-contained builder+relay service. Note: This single shared-builder mode is deprecated; use buildoor_params.instances for dedicated builders instead.
    • mock: A lightweight mode that only brings up a mock-builder and mev-boost for every EL/CL pair.

    Flashbots Implementation Details:

    • Validators register with the relay automatically after the 1st epoch.
    • The relay begins receiving execution payloads from the builder after the 3rd epoch.
    • Validators start receiving validated execution payload headers via mev-boost after the 4th epoch.
    kurtosis run github.com/ethpandaops/ethereum-package '{"mev_type": "flashbots"}'

  6. Quickstart: Run the Ethereum Package with default configuration

    main

    To spin up a private Ethereum testnet locally using Docker, ensure you have Docker and the Kurtosis CLI installed, then run the package using the following command. This will generate genesis information, bootstrap nodes, and start monitoring tools like Grafana and Prometheus.

    kurtosis run --enclave my-testnet github.com/ethpandaops/ethereum-package

  7. Install the kurtosis-ethereum AI agent skill

    main

    The kurtosis-ethereum skill allows AI coding agents (Claude Code and OpenAI Codex) to spin up and manage Ethereum devnets using natural language.

    To use this skill with Claude Code, you must manually copy the skill directory from the cloned repository to your local Claude skills folder. Once copied, Claude Code will auto-discover it, and you can invoke it using the /kurtosis-ethereum command.

    Codex users do not need to perform any installation; the skill is automatically discovered from the .agents/skills/ directory when working within this repository.

    git clone https://github.com/ethpandaops/ethereum-package.git
cp -r ethereum-package/.claude/skills/kurtosis-ethereum ~/.claude/skills/

  8. Inject local EL/CL binaries into containers (Docker only)

    main

    For rapid debugging, you can inject locally compiled binaries into the containers instead of using standard Docker images.

    Requirements & Constraints:

    1. Location: The binary file must live inside the ethereum-package directory.
    2. Compatibility: The binary must be compiled on a Linux system with libraries matching the client's Dockerfile.
    3. Workflow: Build the client in its own repository, then copy only the final binary to ethereum-package/binaries/.
    4. Restart: You must set el_force_restart: true (for EL) or cl_force_restart: true (for CL) to ensure the container is recreated with the new binary.

    Example (Reth):

    # From the reth repo
cargo build --release --bin reth && cp target/release/reth ../ethereum-package/binaries/

    Configuration:

    el_binary_path: "./binaries/reth"
el_force_restart: true

  9. Configure PeerDAS with pre-generated node keys

    main
    To achieve a perfect column distribution on a 128-column network with an 8-column custody requirement, you can use a specific set of 16 nodes. Each node is configured to custody 8 unique columns. The required private keys are located in the static_files/peerdas-node-keys directory of the repository.

  10. Configure Shadowforking for public networks

    main

    To enable shadowforking, set the network_params.network parameter to the target network name followed by -shadowfork (e.g., hoodi-shadowfork). Note that the persistent: true parameter must also be enabled for shadowforks to function. On Kubernetes, this is currently limited to single-node clusters.

    network_params:
  network: "hoodi-shadowfork"
persistent: true

  11. Download genesis files from the enclave

    main

    You can retrieve network genesis files (like EL genesis data) from a running enclave using the files download command.

    # Example: Retrieve EL genesis data to your Downloads folder
kurtosis files download my-testnet el-genesis-data ~/Downloads