Boxo SDK

repository·main·Indexed 43 days ago

https://github.com/ipfs/boxo

A Go-based SDK and component library for building IPFS applications. It provides reusable modules for content routing, data transfer, and naming, including the components that power the Kubo IPFS implementation. Key features include the AutoConf client for network configuration and the Bitswap exchange for block retrieval and trading.

Tokens
23.7K
Snippets
81
Records
128
Agent score
56%

What's inside Boxo

  1. What is Boxo?

    main

    Boxo is an SDK and component library for building IPFS applications and implementations in Go. It provides high-quality, battle-tested components used by major projects like Kubo (the primary IPFS implementation) and Lotus (the Filecoin implementation).

    Use Boxo if you are:

    • Building an application that interacts with the IPFS network.
    • Building a custom IPFS implementation.
    • Reusing specific IPFS components like Kademlia DHT, Bitswap, or data encoding.
    • Experimenting with IPFS protocols in a Go environment.
  2. Use Routing V1 Server and Client over HTTP API

    main

    Boxo provides a Delegated Routing V1 implementation that operates over an HTTP API. This allows for routing operations to be performed by a server, which can be consumed by a client.

    For implementation details and API signatures, refer to the Go documentation. For the underlying protocol and data formats, refer to the Routing V1 Specification.

  3. Use boxo/files for IPFS file interfaces and utilities

    main

    The boxo/files package provides file interfaces and utility functions used across various Go implementations of IPFS. It is designed to provide a common set of abstractions for handling file-like data within the IPFS ecosystem.

    For detailed API documentation, including specific interface definitions and function signatures, refer to the official Go package documentation.

    https://pkg.go.dev/github.com/ipfs/boxo/files
  4. How the AutoConf client handles errors and fallbacks

    main

    The client implements a multi-tier graceful fallback strategy to ensure high availability of configuration data:

    1. Remote Fetch: The client first attempts to fetch the latest autoconf.json from the configured remote URL(s).
    2. Cache Fallback: If the remote fetch fails (e.g., network error, timeout), the client falls back to the latest successfully cached version on disk.
    3. Explicit Fallback: If no cache exists, the client uses the fallback function provided via WithFallback.

    Note on Method Behavior:

    • GetCached() and GetCachedOrRefresh(ctx) are designed to never fail; they will always return a usable configuration via one of the fallback tiers.
    • GetLatest(ctx) is for explicit control and will return errors if the network fetch fails, allowing the caller to handle the error manually.
  5. Configure Bitswap Broadcast Reduction

    main

    Bitswap Broadcast Reduction is a feature that reduces the number of broadcast messages sent to peers, lowering bandwidth usage during load spikes. Instead of broadcasting to all peers, the client only sends broadcasts to non-local peers if they have previously indicated they possess the requested data blocks.

    To use this feature, you must enable BroadcastControlEnable. When enabled, you can fine-tune the behavior using several configuration options to balance bandwidth savings against data retrieval performance.

    // Refer to bitswap/client/example_test.go for a full implementation example
  6. Access the CAR File Gateway via Subdomain or Path modes

    main

    Once the gateway is running, you can access content using two different gateway patterns:

    Subdomain gateway

    Accessible at http://localhost:8040. This behaves like a standard Subdomain IPFS Gateway, but content is strictly limited to what is contained within the provided CAR file. Note: IPNS resolution and fetching content from the wider IPFS network will not work in this mode.

    Path gateway

    If you do not require Origin isolation and only need to host flat files, you can use a plain path gateway at http://127.0.0.1:8040.

  7. Understand the FailureError model

    main

    The FailureError model is used to represent error states in the pinning remote client. It consists of a mandatory Reason and an optional Details field.

    Properties

    • Reason (string): A mandatory string that identifies the specific type of error.
    • Details (*string): An optional, longer description of the error. This field is intended for supplementary information such as transaction UUIDs for support or links to documentation.
  8. Understand the Bitswap implementation components

    main

    The Bitswap implementation is composed of several specialized managers that coordinate the exchange of blocks:

    • Bitswap (Class): The main entry point that implements the Exchange API. It receives incoming messages, records statistics, and informs the Engine and SessionManager of new wants or received blocks.
    • Engine: Manages the PeerTaskQueue and the Ledger. It converts wants and received blocks into tasks (like HAVE, block, or DONT_HAVE) and periodically pops them to send messages to peers.
    • SessionManager: Coordinates between the BlockPresenceManager (tracking who has what), the Sessions (managing groups of requests), and the PeerManager.
    • PeerManager: Maintains a MessageQueue for every connected peer. It tracks which wants have been sent to which peer and handles timeouts.
    • ProviderQueryManager: Handles fallback discovery by querying the DHT when connected peers do not have the requested blocks. It includes rate-limiting and request deduplication.
    • BlockPresenceManager: Tracks which peers have sent HAVE or DONT_HAVE messages for specific blocks.
  9. What components are included in Boxo?

    main

    Boxo provides a cohesive interface for several core IPFS functional areas, including:

    • Content routing: DHT, delegated content routing, and providing.
    • Data transfer: Gateways, Bitswap, and incremental verification.
    • Naming and mutability: Name resolution and IPNS.
    • Network interaction: Tools for interacting with both public and private IPFS networks.
    • Content-addressed data: Utilities for working with content-addressed data structures.
  10. Optimize related block requests using Bitswap Sessions

    main

    When requesting blocks that are part of the same MerkleDAG or are likely to come from the same peers, use a Bitswap Session. Sessions allow Bitswap to perform internal optimizations to fetch related blocks faster.

    To use a session:

    1. Call exchange.NewSession(ctx) to create a session.
    2. Use the session's GetBlocks (or GetBlock) method to make requests.
    3. Subsequent requests made through the same session will benefit from the optimization.

    The session object implements an interface with the same method signatures as the main Bitswap exchange.

    var ctx context.Context
    var cids []cids.cid
    var exchange bitswap.Bitswap
    
    session := exchange.NewSession(ctx)
    blocksChannel, err := session.GetBlocks(ctx, cids)
    // later
    var relatedCids []cids.cid
    relatedBlocksChannel, err := session.GetBlocks(ctx, relatedCids)