Boundless

repository·main·Indexed 41 days ago

https://github.com/boundless-xyz/boundless

Core primitives for decentralized proof generation, including smart contracts, a Rust SDK, a CLI, and zkVM guests. Includes Ansible playbooks for deploying the Bento stack (PostgreSQL, Redis, MinIO, agents, REST API) with support for NVIDIA GPU acceleration and Kailua service installation.

Tokens
142.8K
Snippets
428
Records
625
Agent score
51%

What's inside Boundless

  1. Overview of Boundless Indexers

    main

    The boundless-indexer crate provides two distinct indexers that populate a shared PostgreSQL database. These indexers allow you to track protocol activity and reward distributions.

    1. Market Indexer: Monitors Boundless Market protocol events (such as proof requests and fulfillments) across any EVM-compatible chain where the protocol is deployed.
    2. Rewards Indexer: Specifically monitors Ethereum mainnet (or Sepolia) for ZKC token staking, PoVW (Proof of Verifiable Work) rewards, and delegation data.

    Both indexers use the same database schema and can be run independently or concurrently.

  2. Overview of the Rewards crate

    main
    The rewards crate provides utility functions designed to fetch rewards data directly from the blockchain and pre-process it for consumption by downstream applications. It is intended for use in tools such as explorers, indexers, and Jupyter notebooks that require structured reward information.
  3. Market Indexer functionality and data

    main

    The Market Indexer monitors the Boundless Market protocol contract to track the lifecycle of requests and market statistics. It supports any EVM-compatible chain where the Boundless Market contract is deployed.

    Tracked Data includes:

    • Proof Request Submissions: Both onchain (via contract events) and offchain (via the order stream API).
    • Request Locking Events: When provers lock requests for fulfillment.
    • Proof Deliveries and Fulfillments: The completion of requests.
    • Request Statuses: Lifecycle states such as submitted, locked, fulfilled, and expired.
    • Cycle Counts: Computed for fulfilled requests based on the requestor address.
    • Market Aggregates: Precomputed summaries (hourly, daily, weekly, monthly, and all-time) for the whole market, individual requestors, and individual provers. These include total requests, fulfillments, fees, rewards, and computational work.
  4. How the Prover architecture works

    main

    The Prover is a distributed zero-knowledge proof (ZKP) computation platform built on RISC Zero's zkVM. It breaks ZKP workloads into parallelizable tasks and distributes them across CPU and GPU worker nodes coordinated through Redis.

    Core Components

    • REST API (crates/api): The central HTTP service. It accepts job submissions, manages image/input uploads, orchestrates the task DAG in Redis, and serves results. It also provides internal endpoints for GPU agents to claim tasks.
    • Workflow Agent (crates/workflow): A single binary that runs in different modes (defined by --task-stream) to perform specific parts of the proof pipeline.
    • TaskDB (crates/taskdb): A Redis-backed task scheduler that handles dependency resolution, priority queues (high/medium/low), work claims, retries, and timeouts.
    • Bento Client (crates/bento-client): A CLI tool used for submitting test jobs, benchmarking, and smoke testing.

    Worker Connectivity Model

    • CPU Workers (exec, aux): Connect directly to Redis to read/write tasks.
    • GPU Workers (prove, join, snark, keccak): Operate statelessly through the REST API; they do not require direct Redis access.
  5. The Prover proof pipeline lifecycle

    main

    A job submission follows a specific Task Directed Acyclic Graph (DAG):

    1. Execute: Runs the guest program to produce $N$ segments.
    2. Prove: Proves each segment independently in parallel.
    3. Join: Performs a binary-tree reduction of proofs ($\log_2(N)$ levels).
    4. Resolve: Resolves assumption-based verification.
    5. Finalize: Produces the final composite receipt.
    6. Snark (optional): Compresses the result to Groth16 or BLAKE3-Groth16.

    Note on POVW: If the POVW_LOG_ID environment variable is set, the join and resolve steps use Proof of Verifiable Work (POVW)-aware variants that track proof provenance.

  6. Understand the Market Indexer architecture

    main

    The Market Indexer is designed around three core principles:

    1. Denormalized table structure: Uses a request_status table as a flattened view of data from proof_requests, event tables, proofs, and cycle_counts to avoid expensive joins during frontend queries.
    2. Precomputed aggregations: Summary statistics (market-wide, per-requestor, and per-prover) are precomputed into dedicated tables for fast analytics.
    3. Event sourcing pattern: Raw blockchain events are stored in *_events tables, allowing the request_status materialized view to be rebuilt if necessary.

    The system runs three concurrent tasks orchestrated by IndexerService::run():

    • Core Loop (main thread): Fetches blockchain logs, processes events (e.g., RequestSubmitted, ProofDelivered), updates the request_status table, and tracks progress in last_block (id=0).
    • Aggregation Supervisor (spawned task): Computes market, requestor, and prover statistics across five time periods (Hourly, Daily, Weekly, Monthly, All-Time). It reads from last_block (id=0) and writes to last_aggregation_block (id=1).
    • Execution Supervisor (optional spawned task): Manages cycle count computations. For unknown requestors, it submits execution requests to an external Bento API and polls for results.
  7. Configure AWS Authentication for Vector

    main

    Vector supports three authentication methods for CloudWatch Logs. They are resolved in the following order:

    1. IAM Instance Profile (Preferred for EC2)

      • Set vector_aws_use_instance_profile: true (default).
      • Requires an IAM role attached to the EC2 instance with permissions for logs:CreateLogGroup, logs:CreateLogStream, logs:DescribeLogGroups, logs:DescribeLogStreams, and logs:PutLogEvents.
    2. AWS Credentials File

      • Set vector_aws_credentials_file to the path of your credentials file (e.g., "/root/.aws/credentials").
    3. Environment Variables

      • Set vector_aws_access_key_id and vector_aws_secret_access_key directly. These are written to /etc/default/vector.
  8. How the Redshift Telemetry Schema works

    main

    The Redshift schema is designed in two layers to decouple data ingestion from data querying, allowing for easy schema evolution.

    Layer 1: Raw Materialized Views

    • Names: heartbeats_raw, evaluations_raw, completions_raw.
    • Function: Stores the full JSON payload from Kinesis in a single Redshift SUPER column named data.
    • Stability: These views are stable and do not require schema changes when telemetry fields change.

    Layer 2: Typed SQL Views

    • Names: broker_heartbeats, request_evaluations, request_completions.
    • Function: Extracts and casts individual fields from the data column using dot notation (e.g., data.field_name::TYPE). They also handle record deduplication using ROW_NUMBER().
    • Behavior: If a typed view references a field that is missing in a record (e.g., older historical data), Redshift returns NULL for that column instead of erroring.
  9. How Bento's distributed architecture works

    main

    Bento is a distributed ZKP computation platform that uses a worker-based architecture to scale RISC Zero guest program execution. The system is composed of several key layers:

    • Workflow Engine (crates/workflow): The orchestrator. It manages task distribution, worker load balancing, and retries. It uses different Worker Types to handle specific stages of the ZKP lifecycle:
      • exec: Executes guest programs and generates segments.
      • prove: Generates STARK proofs from segments.
      • join: Composes multiple proofs into larger proofs.
      • resolve: Handles assumption-based verification.
      • finalize: Finalizes proof verification.
      • snark: Converts STARK proofs to SNARKs.
      • keccak: Handles Keccak hash computations.
      • union: Combines multiple proof types.
    • Task Database (crates/taskdb): A PostgreSQL-based system that tracks job/task lifecycles (Pending, Ready, Running, Done, Failed) and manages dependencies.
    • API Service (crates/api): The entry point for users to submit jobs and manage the cluster via HTTP.
    • Client Library (crates/bento-client): A Rust library for programmatic interaction with the API.
  10. Monitor Log Errors via CloudWatch Metrics

    main

    When vector_track_log_errors: true (default), Vector counts journald log lines containing the string "ERROR" and publishes a counter to CloudWatch Metrics.

    • Metric: bento_log_errors (incremented by 1 for each ERROR line from monitored units).
    • Usage: Use this for error-rate dashboards and alarms (e.g., monitoring the Sum over a 5-minute period).
  11. Smart Contract Relationship Diagram

    main

    The following architecture describes the relationship between the Boundless Market proxy, the verifier router, and the emergency stop mechanisms:

    ---
    title: Smart Contract Relationships
    ---
    flowchart LR
      subgraph IRiscZeroVerifier
        subgraph routers["Routers"]
            router["RiscZeroVerifierRouter [managed]"]
        end
    
        subgraph emergencyStops["Emergency Stop Proxies"]
          groth16v1ES["RiscZeroVerifierEmergencyStop"]
          groth16v2ES["RiscZeroVerifierEmergencyStop"]
          setv1ES["RiscZeroVerifierEmergencyStop"]
          setv2ES["RiscZeroVerifierEmergencyStop"]
        end
    
        subgraph impls["Base Implementations"]
            groth16v1["RiscZeroGroth16Verifier [v1]"]
            groth16v2["RiscZeroGroth16Verifier [v2]"]
            setv1["RiscZeroSetVerifier [v1]"]
            setv2["RiscZeroSetVerifier [v2]"]
        end
    
        router -- calls --> groth16v1ES
        router -- calls --> groth16v2ES
        router -- calls --> setv1ES
        router -- calls --> setv2ES
    
        groth16v1ES -- calls --> groth16v1
        groth16v2ES -- calls --> groth16v2
        setv1ES -- calls --> setv1
        setv2ES -- calls --> setv2
      end
      subgraph Boundless market
        subgraph proxy["ERC1967Proxy"]
            marketProxy["BoundlessMarket [proxy]"]
        end
        subgraph BoundlessMarket["Boundless market implementations"]
            boundlessMarketv1["BoundlessMarket [v1]"]
            boundlessMarketv2["BoundlessMarket [v2]"]
        end
    
        marketProxy -- calls --> boundlessMarketv2
      end
      timelock[TimelockController]
      multisig["RISC Zero Multisig"]
    
      timelock -- admin --> router
      multisig -- proposer --> timelock
      multisig -- guardian --> emergencyStops
      multisig -- upgrade --> proxy
      setv1 -- calls --> router
      setv2 -- calls --> router
      boundlessMarketv2 -- calls --> router