Celestia Node

The Celestia Node is built around a central Node data structure that manages various modular services. Services are processes for data retrieval, sharing, or storage that can be started and stopped on the core node.

Services available to Light Nodes

• ExtendedHeaderService: Handles retrieving, broadcasting (ExtendedHeaderSub), and storing ExtendedHeaders via ExtendedHeaderExchange.
• FraudProofService (optional): Handles retrieving BadEncodingFraudProofs and StateFraudProofs via FraudProofSub and FraudProofStore.
• ShareService: Handles retrieving shares via sampling or namespace via ShareExchange and ShareStore.
• StateService (optional): Handles retrieving state for a block height or account via StateExchange.
• TransactionService (optional): A server handling endpoints like /submit_tx via SubmitTx.

Services available to Full Nodes

Full nodes include all light node services with the following specialized implementations:

• ExtendedHeaderService: Adds generation and verification (ExtendedHeaderVerification) capabilities.
• FraudProofService: Adds FraudProofGeneration.
• ShareService: Uses ShareExchange but does not use a separate ShareStore (as it stores full blocks instead).
• BlockService: Manages BlockErasureCoding, NewBlockEventSubscription (from Celestia Core), BlockExchange (between full nodes), and BlockStore.
• StateService & TransactionService: Same as light nodes.

A Celestia Node can be initialized in two modes, each providing different capabilities and services:

Light Node

Designed for Data Availability (DA) sampling. It performs the following:

• Verifies ExtendedHeaders.
• Propagates relevant block information (ExtendedHeaders and BadEncodingFraudProofs) to peers.
• Performs and serves sampling and SharesByNamespace requests.
• Requests State to retrieve AccountBalance for transaction submission.

Full Node

Includes all light node capabilities plus additional services for block processing and storage:

• Receives "raw" (un-erasure coded) blocks from a Celestia Core node via NewBlockEvents (using the /block RPC endpoint).
• Performs erasure coding and verifies erasure coding.
• Generates ExtendedHeaders (containing the raw block header, DataAvailabilityHeader (DAH), and ValidatorSet) and serves them to the network.
• Can optionally request raw blocks from other full nodes.

A Bad Encoding Fraud Proof (BEFP) is generated by a Full Node when it detects ErrByzantineData from the rsmt2d library during block reparation (i.e., when recovered data does not match its respective row/column roots).

When a BEFP is detected:

1. The Full Node generates a BadEncodingProof containing the Height, the Index of the failed row/column, the Axis (ROW or COL), and the Shares (including their MerkleProof).
2. The Full Node broadcasts this proof to the network via a Broadcaster using libp2p pubsub.
3. Light and Full nodes subscribe to the BEFP topic to receive and verify these proofs.
4. Upon receiving a valid BEFP, nodes must stop dependent services: DAS, Syncer, and SubmitTx.
5. Valid BEFPs are stored on disk using the FraudStore interface under the fraud/badEncodingProof path.

// Example of the BadEncodingProof structure
type BadEncodingProof struct {
    Height uint64
    Shares []*ShareWithProof
    Index  uint8
    Axis   rsmt2d.Axis
}

The SHREX protocol defines two primary node types with distinct responsibilities:

Light Nodes (LN)

Light nodes are consumers of data. Their responsibilities include:

• Subscribing to ShrEx/Sub notifications to learn about new blocks.
• Using Peer Discovery (DHT) to find bridge nodes.
• Using the SHREX Client to request specific data.
• Note: Light nodes do not advertise themselves or serve data to others.

Bridge Nodes (BN)

Bridge nodes are providers of data. Their responsibilities include:

• Acting as ShrEx/Sub Publishers to announce new block availability.
• Using the Discovery Service to advertise their presence.
• Acting as a SHREX Server to serve requested data.
• Storing block data (both recent and historical).
• Advertising: Bridge nodes advertise themselves using the full tag (or the archival tag if the pruner is disabled).

┌─────────────────────────────────────────────────────────────┐
│                        Light Node                           │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌────────────────────────────────────────────────────┐     │
│  │              Peer Manager                          │     │
│  │  - Maintains validated peer pools                  │     │
│  │  - Selects optimal peers for requests              │     │
│  │  - Validates ShrEx/Sub notifications               │     │
│  └──────┬──────────────────────┬──────────────────┬───┘     │
│         │                      │                  │         │
│         │                      │                  │         │
│  ┌──────▼────────┐    ┌────────▼────────┐  ┌─────▼──────┐   │
│  │  ShrEx/Sub    │    │   Discovery     │  │  SHREX     │   │
│  │  Subscriber   │    │   Service       │  │  Client    │   │
│  │               │    │                 │  │            │   │
│  └───────────────┘    └─────────────────┘  └────────────┘   │
│                                                             │
└─────────────────────────────────────────────────────────────┘
                              │
                              │ Network
                              │
┌─────────────────────────────────────────────────────────────┐
│                    Bridge Node                              │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌────────────────┐    ┌──────────────┐  ┌──────────────┐   │
│  │  ShrEx/Sub     │    │  Discovery   │  │   SHREX      │   │
│  │  Publisher     │    │  Service     │  │   Server     │   │
│  │                │    │              │  │              │   │
│  └────────────────┘    └──────────────┘  └──────────────┘   │
└─────────────────────────────────────────────────────────────┘

The Peer Manager is the central coordination component for peer selection in the SHREX protocol. It manages two types of peer sources to ensure efficient data retrieval:

1. ShrEx/Sub Notifications: Peers announcing specific data availability via pubsub. These are placed into Data Hash Pools and must be validated against headers before being promoted.
2. Discovery Service: Peers found via DHT-based discovery. These are added directly to the Discovered Nodes Pool.

Peer Selection Priority

When requesting data, the manager follows this priority order:

1. First Priority: Peers from validated data-hash-specific pools (highest confidence).
2. Second Priority: Peers from the general Discovered Nodes Pool.
3. Blocking: If no peers are available, the system waits subject to a context timeout.

Peer Pool Lifecycle

• Data Hash Pools: Created when a ShrEx/Sub notification arrives for a specific hash. Peers are added to these pools and held until a matching header is received via the Header Subscription. Once validated, peers are promoted to the Discovered Nodes Pool.
• Discovered Nodes Pool: A general pool containing peers from the Discovery service and promoted peers from validated Data Hash Pools.

/* Peer Selection Priority Logic */
1. Validated data-hash-specific pools
2. Discovered nodes pool
3. Blocking (subject to context timeout)

The celestia-node uses two primary abstractions to manage interaction with the Celestia network state and transaction submission:

1. StateService: A high-level service that manages RPC endpoints. It acts as the entry point for users to access state-related methods.
2. StateAccessor: An interface that defines how the node interacts with celestia-core to retrieve account information and submit transactions.

Both light and full nodes run a StateService. The StateAccessor interface allows for different underlying implementations (CORE, P2P, or LOCAL) without changing the high-level service logic.

type StateService struct {
   accessor StateAccessor
}

type StateAccessor interface {
   // Balance retrieves the Celestia coin balance for the node's account/signer.
   Balance(ctx context.Context) (*Balance, error)
   // BalanceForAddress retrieves the Celestia coin balance for the given address.
   BalanceForAddress(ctx context.Context, addr types.AccAddress) (*Balance, error)
   // SubmitTx submits the given transaction/message and blocks until included in a block.
   SubmitTx(ctx context.Context, tx Tx) (*TxResponse, error)
}