DOCSEARCHbyAlgolia
Submit Docs

celestia-core Documentation

repository·main·Indexed 19 days ago

https://github.com/celestiaorg/celestia-core

A specialized fork of CometBFT designed for the Celestia network. It modifies the consensus engine to support data availability sampling via custom DataHash calculations, compact block propagation, and extended gRPC APIs. The repository includes documentation on the Application BlockChain Interface (ABCI), cryptographic interfaces, and the KVStoreApplication example, while leveraging the Tendermint consensus algorithm for Byzantine Fault Tolerance (BFT).

Tokens
241K
Snippets
474
Records
998
Agent score
68%

What's inside celestia-core

  1. Overview of Ivy Proof specifications

    main

    The spec/ivy-proofs directory contains formal specifications and proofs of the Tendermint consensus algorithm written in Ivy. The files are organized as follows:

    • tendermint.ivy: A specification of the Tendermint algorithm based on The latest gossip on BFT consensus.
    • abstract_tendermint.ivy: A verification-friendly, abstract version of the Tendermint specification.
    • classic_safety.ivy: A proof of the classic BFT safety property (no two well-behaved nodes ever disagree if quorums overlap with a well-behaved node).
    • accountable_safety_1.ivy: A proof that if two well-behaved nodes disagree, evidence exists demonstrating at least $f+1$ nodes misbehaved (assuming every quorum has at least one well-behaved node).
    • accountable_safety_2.ivy: A proof that well-behaved nodes cannot be framed by malicious nodes (malicious nodes cannot construct evidence to incriminate well-behaved nodes).
    • network_shim.ivy: The network model and a shim object for interfacing with the Tendermint specification.
    • domain_model.ivy: The underlying domain model (rounds, values, quorums, etc.) used by the specifications.

  2. Overview of the CometBFT JSON-RPC specification

    main
    The CometBFT RPC specification defines the JSON-RPC 2.0 interface used by clients to interact with the network. It supports multiple transport protocols including HTTP, HTTPS, and WebSockets (WS). This specification is intended to be implemented by all clients to ensure interoperability with CometBFT nodes.

  3. Overview of CometBFT v0.38.x Protocol Buffers Definitions

    main

    CometBFT v0.38.x uses Protocol Buffers (Protobuf) definitions to define types used across several core layers of the system:

    • Application Blockchain Interface (ABCI): Specifically for remote applications.
    • P2P Layer: For node-to-node network interactions.
    • Remote Signers (privval): For interactions with remote signing modules.
    • RPC: Used for native JSON serialization when handling RPC requests and responses.
    • Storage Layer: For data serialization and deserialization to on-disk storage.

    Note: While v0.x releases use tendermint Protobuf definitions to ease the transition from Tendermint Core, CometBFT v1 and later will transition to a dedicated cometbft package of definitions.

  4. Overview of the CometBFT Blockchain implementation

    main

    The CometBFT protocol implementation within celestia-core provides the core logic for blockchain operations. The implementation is organized into three primary functional areas:

    1. Core Data Structures: Defines the fundamental types used throughout the protocol.
    2. Encoding: Handles the serialization and deserialization of data for network transmission and storage.
    3. State: Manages the blockchain state and transitions.

  6. Overview of the CometBFT p2p layer implementation

    main

    The peer-to-peer (p2p) communication layer in CometBFT is organized around several core abstractions that manage secure connections, peer discovery, and message routing. The implementation follows the structure of the p2p package and is composed of the following primary components:

    • Transport: Handles the establishment of secure and authenticated connections with peers.
    • Switch: Acts as the central routing hub. It is responsible for dialing peers, accepting incoming connections, managing established connections, and routing messages between local and remote CometBFT protocol reactors.
    • PEX (Peer Exchange) Reactor: A multi-faceted component responsible for peer discovery and management, subdivided into:
      • Peer Exchange protocol: Facilitates the exchange of peer addresses between nodes.
      • Address Book: A storage mechanism for discovered peer addresses and quality metrics for peers.
      • Peer Manager: Logic that determines when and to which peers the node should initiate outbound connections.
    • Types and Configuration: Defines the data structures and parameters used to tune the p2p layer.

  7. Overview of Reporting Scripts

    main

    The reporting scripts directory provides utilities for QA and performance reporting:

    • latency_throughput.py: Plots transaction latency vs throughput from loadtime tool CSV output.
    • latency_plotter.py: Advanced version of the latency plotter that handles multiple experiments and configurations, generating grouped subplots.
    • prometheus_plotter.py: Plots various metrics extracted from a Prometheus instance.

  8. What is CometBFT

    main

    CometBFT is a software package designed for the secure and consistent replication of an application across multiple machines. It provides Byzantine Fault Tolerance (BFT), meaning the system remains functional and consistent as long as less than 1/3 of the machines fail or behave maliciously.

    CometBFT consists of two main components:

    1. Consensus Engine: Based on the Tendermint consensus algorithm, it ensures all machines record the same transactions in the exact same order.
    2. Application BlockChain Interface (ABCI): A generic interface that delivers transactions to an application for processing.

    Unlike monolithic blockchain solutions, CometBFT decouples the consensus and P2P layers from the application state, allowing developers to write their application logic in any programming language.

  9. Understand the Rotating Node Testnet experiment results

    main

    The Rotating Node Testnet experiment evaluates CometBFT performance under a specific workload defined as c=1,r=400. This workload is considered a safe level as it remains below the saturation point observed in larger 200-node testnets.

    Key characteristics of this testbed include:

    • Node Composition: 10 validators and 25 full nodes.
    • Baseline Comparison: Results are compared against v0.37.0-alpha.2 (Tendermint Core).
    • Workload Behavior: Unlike some previous versions, transactions in this test are not rejected by CheckTx, resulting in a higher actual transaction load and higher average Resident Set Size (RSS) compared to baselines where transactions were rejected.

    Performance Metrics Observed

    • Latencies: Average increase of approximately 1 second compared to the baseline due to higher transaction throughput.
    • Blocks and Transactions: Block production rates remain similar to the baseline, but the transactions processed per minute are significantly higher because CheckTx does not reject the load.
    • Peers: Peer evolution patterns are comparable to the baseline.
    • Memory (RSS): The average Resident Set Size is notably larger in v0.38.0-alpha.2 due to the higher volume of processed transactions.
    • CPU Utilization: Measured via the load1 metric; utilization typically stays under 5, which is considered normal load.

  10. Understand the CometBFT Quality Assurance process

    main

    The CometBFT Quality Assurance (QA) process is used to validate releases before they are cut. The QA process involves running specific tests and comparing results against a baseline (typically a previous stable version of Tendermint Core or CometBFT) to ensure stability and performance.

    Detailed reports for specific releases are maintained in this directory, documenting the tests performed and the comparison with the baseline. If you are looking for the specific steps taken during QA, refer to the method documentation.

  11. Understand CometBFT v0.38.x QA Results and Changes

    main

    The QA report for CometBFT v0.38.0-alpha.2 highlights the transition to ABCI 2.0. The primary functional change compared to the v0.37.x baseline is the introduction of the FinalizeBlock method, which completes the full range of ABCI++ functionality.

    Key issues addressed in this version include:

    • Proposer crashes in PrepareProposal: Fixed a critical bug where proposers lacked necessary extensions, particularly during catch-up.
    • Metrics-related panics: Fixed critical bugs in metrics logic that caused panics during testnet startup.

  12. Use Cases for P2P Light Clients

    main

    The P2P light client implementation targets specific operational needs:

    1. IBC Relayers: Provides higher reliability and lower operational costs for relayer operators (like those using Hermes). It removes the choice between using unreliable public RPC endpoints or the high cost of running a full node.
    2. Non-Go Clients: By leveraging libp2p for service discovery and connection handling, non-Go implementations (such as Rust-based clients) can more easily integrate with the P2P network without reimplementing the entire Tendermint P2P stack.
    3. Ephemeral Client Use: While P2P is intended for more robust connections, client-side users (like wallets) may still prefer using a subset of the RPC for ephemeral or lightweight interactions.