DOCSEARCHbyAlgolia
Submit Docs

hyperdht

repository·main·Indexed 19 days ago

https://github.com/holepunchto/hyperdht

A Distributed Hash Table (DHT) implementation that powers Hyperswarm. It facilitates peer discovery and encrypted P2P connectivity using UDP holepunching techniques. The library provides capabilities for creating P2P servers, managing immutable and mutable records, and performing peer lookups. It includes a CLI for running DHT nodes and bootstrap nodes, as well as a handshake and holepunching process to navigate firewalls.

Tokens
8.5K
Snippets
31
Records
37
Agent score
63%

What's inside hyperdht

  1. How the handshake process works in hyperdht

    main

    A handshake establishes a connection between a client node and a server node using two intermediary relay nodes: one chosen by the client and one chosen by the server. The client and server do not interact directly during this phase; instead, they communicate through these relays to navigate firewalls.

    The Handshake Workflow

    1. Discovery: The client issues a DHT query to find the server node. If the chosen client relay doesn't know the server, it forwards the query through the DHT until the server is located.
    2. Initiation: Once the server and its relay are identified, the client sends a PEER_HANDSHAKE command to the server's relay node.
    3. Relay Chain: The server's relay forwards the command to the server, which then forwards it to the client's relay, which finally replies to the client.
    4. Optimization: If a relay node is also the server node, it skips the intermediate steps and replies directly.

    Resulting State

    After a successful handshake:

    • The client node maintains firewall sessions with its own relay and the server's relay.
    • The server node maintains firewall sessions with its own relay and the client's relay.
    • The client node obtains the server's address, allowing it to begin direct holepunching.

  2. How the Holepunch process works

    main

    Holepunching is the process of establishing a direct peer-to-peer connection between a client and a server after a handshake has completed.

    1. Pre-check: The client first checks if a direct connection already exists (e.g., if the server acted as the relay node during the handshake).
    2. Signaling: If no connection exists, the client and server exchange PEER_HOLEPUNCH commands via relay nodes to coordinate the punch.
    3. Verification: The process uses tokens (token and remoteToken) to allow both peers to verify that the incoming connection address is correct and belongs to the intended peer.
    4. Pinging: Simultaneously with the signaling messages, both peers attempt to ping the perceived address of the other. The holepunch messages provide feedback regarding network conditions and additional addresses to attempt.
    5. Completion: The process concludes once both peers have successfully received a ping from the other, establishing a direct connection.

  4. How to create a P2P server and connect to it

    main

    HyperDHT allows you to create encrypted P2P servers that use UDP holepunching to work on most networks. To establish a connection, one node creates a server listening on a specific keyPair, and the other node connects using the server's publicKey.

    import DHT from 'hyperdht'

// --- Server Side ---
const node = new DHT()
const server = node.createServer()
const keyPair = DHT.keyPair()

server.on('connection', function (socket) {
  console.log('Remote public key', socket.remotePublicKey)
  process.stdin.pipe(socket).pipe(process.stdout)
})

await server.listen(keyPair)

// --- Client Side ---
// Use keyPair.publicKey from the server side to connect
const socket = anotherNode.connect(publicKey)

socket.on('open', function () {
  // socket fully open with the other peer
})

process.stdin.pipe(socket).pipe(process.stdout)

  5. Set up an isolated DHT network

    main

    To run a private network, you must start your own bootstrap node and then point all other nodes to it. This prevents your nodes from joining the public mainnet.

    # 1. Start the first bootstrap node
hyperdht --bootstrap --host (server-ip) --port 49737

# 2. Start subsequent nodes pointing to your bootstrap node
hyperdht --port 49738 --bootstrap (server-ip):49737

    In your application code, connect using the bootstrap address:

    const dht = new DHT({ bootstrap: ['(server-ip):49737'] })

  6. Implement a hyperdht Plugin

    main

    To extend hyperdht functionality, you can create a class that extends the Plugin base class. A plugin allows you to intercept or respond to DHT requests and manage persistent state.

    To create a functional plugin, you must implement the following lifecycle and request methods:

    • onregister(dht): Called when the plugin is registered. Use this to store a reference to the dht instance.
    • onrequest(req, outerReq): Called when a plugin request is received. This must be implemented to handle incoming commands.
    • onpersistent(): Called when the DHT requires the plugin to handle persistent data/state.
    • destroy(): Called when the plugin is being shut down. Use this for cleanup.

    Additionally, the base class provides helper methods request() and query() to facilitate communication with other plugins via the DHT using the HYPERDHT_COMMANDS.PLUGIN command.

    const Plugin = require('./lib/plugin')

class MyCustomPlugin extends Plugin {
  constructor() {
    super('my-plugin', '1.0.0')
  }

  onregister(dht) {
    this.dht = dht
  }

  onrequest(req, outerReq) {
    // Handle incoming plugin requests
    console.log('Received request:', req)
  }

  onpersistent() {
    // Handle persistence logic
  }

  destroy() {
    // Cleanup logic
  }
}

  7. Initialize HyperDHT

    main

    To use hyperdht, instantiate the HyperDHT class. It extends dht-rpc and provides enhanced P2P capabilities including peer discovery, record announcement, and mutable/immutable data storage.

    Common configuration options include:

    • port: The port to listen on (defaults to 49737).
    • bootstrap: An array of bootstrap nodes.
    • nodes: An array of initial nodes to connect to.
    • keyPair: A pre-generated keypair. If not provided, a keypair is created from opts.seed.
    • connectionKeepAlive: Interval in ms to keep connections alive (defaults to 5000).
    • randomPunchInterval: Minimum interval between random punches (defaults to 20000).
    const HyperDHT = require('hyperdht')

const dht = new HyperDHT({
  port: 49737,
  seed: 'your-seed-here'
})

  8. Store and retrieve mutable records

    main

    Use these methods to store data that can be updated using sequence numbers (seq).

    // Store a mutable value
const { publicKey, closestNodes, seq, signature } = await node.mutablePut(keyPair, value, [options])

// Retrieve a mutable value
const { value, from, seq, signature } = await node.mutableGet(publicKey, {
  // Only return values with seq >= supplied seq
  seq: 0,
  // If true, tries to find the highest seq before returning
  latest: false
}, [options])

  9. Manage P2P server lifecycle

    main

    Use these methods to control a server instance created via node.createServer():

    • await server.listen(keyPair): Starts listening on the provided keypair. Use keyPair.publicKey as the address for clients to connect.
    • server.refresh(): Reannounces the server's address (called automatically on network changes).
    • server.address(): Returns { host, port, publicKey } of the server.
    • await server.close(): Stops listening.
    • server.on('connection', socket): Emits a NoiseSecretStream instance when a connection passes the firewall.
    • server.on('listening'): Emits when the server is fully listening.
    • server.on('close'): Emits when the server is fully closed.

  10. Discover peers with `node.lookup()`

    main

    Search for peers in the DHT associated with a specific topic. The topic must be a 32-byte buffer (e.g., a hash).

    const stream = node.lookup(topic, options)

// stream emits objects containing:
// from: { id, host, port }
// to: { host, port }
// peers: [ { publicKey, nodes: [{ host, port }, ...] } ]

  11. Create a P2P server with `node.createServer()`

    main

    Creates a server for accepting incoming encrypted P2P connections. The server uses UDP holepunching to facilitate connectivity.

    const server = node.createServer({
  // Validate connections
  firewall (remotePublicKey, remoteHandshakePayload) {
    // return false to reject, true to accept
    return true
  }
}, onconnection)

  12. Connect to a remote server with `node.connect()`

    main

    Connect to a remote server using its public key. This performs UDP holepunching for P2P connectivity. The remotePublicKey can be a buffer, hex string, or z-base32 string.

    const socket = node.connect(remotePublicKey, {
  // Optional array of close dht nodes to speed up connecting
  nodes: [...],
  // Optional key pair to use (defaults to node.defaultKeyPair)
  keyPair
})

socket.on('open', () => {
  // connection established
})

// socket properties:
// socket.remotePublicKey
// socket.publicKey