go-libp2p-pubsub

repository·master·Indexed 18 days ago

https://github.com/libp2p/go-libp2p-pubsub

The canonical implementation of peer-to-peer publish-subscribe protocols for libp2p. It provides multiple routing mechanisms, including Floodsub (baseline flooding), Randomsub (probabilistic routing), and Gossipsub (mesh formation and gossip propagation following the libp2p pubsub spec). The library includes features for internal event tracing (JSON, PB, and Remote tracers), peer restriction via Blacklist interfaces, and topic bootstrapping with readiness checks.

Tokens
21.8K
Snippets
91
Records
113
Agent score
63%

What's inside go-libp2p-pubsub

  1. Understand the available PubSub routers

    master

    The library provides three distinct message router implementations for p2p messaging infrastructure:

    1. Floodsub: The baseline flooding protocol.
    2. Randomsub: A simple probabilistic router that propagates messages to random subsets of peers.
    3. Gossipsub: An advanced router featuring mesh formation and gossip propagation. This is the most sophisticated option and follows the libp2p pubsub spec.
  2. Enable tracing for PubSub internals

    master

    The pubsub system supports tracing to collect all internal events, allowing you to recreate message flows and system states for analysis. To enable tracing, you must instantiate your pubsub system (e.g., using pubsub.NewGossipSub) with the pubsub.WithEventTracer option.

    There are three available tracer implementations:

    • JSON Tracer: Saves traces to a JSON file.
    • PB Tracer: Saves traces to a protobuf file.
    • Remote Tracer: Sends traces to a remote peer (requires a running traced daemon from go-libp2p-pubsub-tracer).
    // Example: Capturing trace as a JSON file
    tracer, err := pubsub.NewJSONTracer("/path/to/trace.json")
    if err != nil {
      panic(err)
    }
    
    ps, err := pubsub.NewGossipSub(ctx, host, pubsub.WithEventTracer(tracer))
  3. Use ValidationResult for granular message decisions

    master

    When using ValidatorEx, you can return one of the following ValidationResult values to control how the network and application handle a message:

    • ValidationAccept (0): The message is valid. It will be delivered to the application and forwarded to the network.
    • ValidationReject (1): The message is invalid. It will not be delivered or forwarded, and the sender should be penalized by peer scoring routers.
    • ValidationIgnore (2): The message should be ignored. It will not be delivered or forwarded, but the sender will not be penalized by peer scoring routers.

    Note: validationThrottled is an internal state used when validation capacity is exceeded.

    const (
    	ValidationAccept ValidationResult = 0
    	ValidationReject ValidationResult = 1
    	ValidationIgnore ValidationResult = 2
    )
  4. Understand the Message and RPC structures

    master

    The pubsub system uses two primary data structures for communication:

    1. Message: Represents the actual payload being published. It embeds *pb.Message and includes metadata like ID, ReceivedFrom, ValidatorData, and a Local flag.
    2. RPC: An envelope used to transport multiple Message objects and Control messages over a stream. It includes a From() method to identify the sender.

    RPC objects can be split into multiple parts if they exceed the maxMessageSize or maxControlMessageSize limits using the split method.

  5. Manage pubsub topics with the Topic type

    master

    The Topic type is the primary handle for interacting with a specific pubsub topic. It allows you to publish messages, subscribe to incoming messages, manage peer events, and configure topic-specific settings like peer scoring.

    Common tasks include:

    • Publishing: Use Publish to send data to the topic.
    • Subscribing: Use Subscribe to receive a stream of messages.
    • Monitoring Peers: Use EventHandler to track peers joining or leaving the topic.
    • Relaying: Use Relay to enable message relaying for the topic.
    • Closing: Use Close to shut down the topic handle.
    // Example of basic topic usage
    // (Assuming a PubSub instance 'ps' is already initialized)
    topic, err := ps.Join("my-topic")
    if err != nil {
        return err
    }
    
    // Publish a message
    err = topic.Publish(ctx, []byte("hello world"))
    
    // Subscribe to messages
    sub, err := topic.Subscribe()
    if err != nil {
        return err
    }
    
    // Read from subscription
    msg, err := sub.Next(ctx)
    if err != nil {
        return err
    }
    fmt.Printf("Received: %s\n", string(msg.Data))
  6. Monitor message delivery with gossipTracer

    master

    The gossipTracer is an internal implementation of the RawTracer interface used to track IWANT requests. It monitors whether peers follow up on IWANT requests after an IHAVE advertisement.

    To prevent excessive memory usage, the tracer uses probabilistic tracking of promises. It identifies 'broken promises'—instances where a peer failed to deliver a requested message within the configured followUpTime—which can then be used to penalize those peers.

    Key lifecycle methods for the tracer include:

    • DeliverMessage(msg *Message): Fulfills a promise when a message is successfully delivered.
    • RejectMessage(msg *Message, reason string): Fulfills a promise when a message is rejected (except for specific signature-related reasons), allowing the score penalty to be applied based on the invalid delivery.
    • ValidateMessage(msg *Message): Fulfills a promise as soon as message validation begins.
    • ThrottlePeer(p peer.ID): Immediately voids all outstanding promises associated with a specific peer, typically called when a peer is being throttled.
  7. Configure MessageSignaturePolicy

    master

    The MessageSignaturePolicy type defines how a pubsub node handles message signatures. It determines whether the node produces signatures for its own messages and whether it expects and verifies signatures on incoming messages.

    There are two primary modes of operation:

    1. Strict Mode: Use StrictSign to ensure all outgoing messages are signed and all incoming messages are verified. Use StrictNoSign to disable signatures entirely; in this mode, the node will drop and penalize incoming messages that attempt to carry a signature.
    2. Lax Mode (Deprecated): LaxSign and LaxNoSign are deprecated. It is recommended to use either strict signing or strict no-signing to avoid ambiguity.

    Note: StrictNoSign is defined as msgVerification in the source, which implies it enforces the verification logic (checking for the absence of signatures) rather than just ignoring them.

    const (
    	// StrictSign produces signatures and expects and verifies incoming signatures
    	StrictSign = msgSigning | msgVerification
    	// StrictNoSign does not produce signatures and drops and penalises incoming messages that carry one
    	StrictNoSign = msgVerification
    )
    
    // Deprecated: it is recommend to either strictly enable, or strictly disable, signatures.
    const (
    	LaxSign = msgSigning
    	LaxNoSign = 0
    )
  8. Use SubscriptionFilter to control topic access

    master

    A SubscriptionFilter is used to decide which topics a node is interested in. It controls both local subscription attempts and incoming subscription notifications from other peers.

    When a SubscriptionFilter is applied:

    1. Joining Topics: If CanSubscribe(topic) returns false, the local Join operation will fail with an error.
    2. Incoming Notifications: When a peer notifies you of its subscriptions via RPC, FilterIncomingSubscriptions is called. If it returns false or filters out a topic, that notification is ignored.

    You can apply a filter to your PubSub instance using the WithSubscriptionFilter option during initialization.

    import (
    	"github.com/libp2p/go-libp2p-pubsub"
    	"github.com/libp2p/go-libp2p"
    )
    
    // Example: Creating a PubSub instance with an allowlist filter
    filter := pubsub.NewAllowlistSubscriptionFilter("topic1", "topic2")
    ps, err := pubsub.New("noise", pubsub.WithSubscriptionFilter(filter))
  9. Define message validators

    master

    You can implement message validation using two types of function signatures:

    1. Validator: A simple binary decision function. Returns true to accept the message or false to reject it.
    2. ValidatorEx: An extended validation function that returns a ValidationResult for more granular control.

    Both types receive a context.Context, the peer.ID of the sender, and a pointer to the *Message being validated.

    // Simple binary validator
    type Validator func(context.Context, peer.ID, *Message) bool
    
    // Extended validator with granular results
    type ValidatorEx func(context.Context, peer.ID, *Message) ValidationResult
  10. Inspect peer scores for debugging

    master

    You can enable periodic inspection of peer scores in a GossipSub router using the WithPeerScoreInspect option. This is useful for debugging how peers are being scored and identifying potential issues with peer behavior or scoring parameters.

    There are two ways to provide an inspector:

    1. Simple Inspection: Provide a function with the signature PeerScoreInspectFn, which receives a map[peer.ID]float64 containing the current score for each peer.
    2. Extended Inspection: Provide a function with the signature ExtendedPeerScoreInspectFn, which receives a map[peer.ID]*PeerScoreSnapshot. This allows you to inspect individual components of the score, such as topic-specific stats, IP colocation factors, and behavioral penalties.

    Note: WithPeerScoreInspect must be passed after the WithPeerScore option in your router configuration.

    // Example using ExtendedPeerScoreInspectFn
    inspector := func(scores map[peer.ID]*pubsub.PeerScoreSnapshot) {
        for id, snapshot := range scores {
            fmt.Printf("Peer %s score: %f (IP Colocation: %f)\n", id, snapshot.Score, snapshot.IPColocationFactor)
        }
    }
    
    // When initializing your PubSub/GossipSub router:
    // ps, err := pubsub.NewGossipSub(host, 
    //     pubsub.WithPeerScore(params), 
    //     pubsub.WithPeerScoreInspect(inspector, 10*time.Second),
    // )
  11. Register default and topic-specific validators

    master

    Validators can be applied globally to all topics or specifically to a single topic using the WithDefaultValidator option during PubSub initialization.

    To register a validator for a specific topic, you typically interact with the validation pipeline (though the direct registration method AddValidator is used internally by the pipeline logic).

    // Example: Adding a default validator to PubSub
    ps, err := pubsub.New(
        // ... other options
        pubsub.WithDefaultValidator(myValidatorFunc),
    )