DATUM Gateway

repository·master·Indexed 41 days ago

https://github.com/ocean-xyz/datum_gateway

A decentralized tool for Bitcoin solo and pooled mining that enables miners to generate block templates locally from their own Bitcoin node. It acts as a bridge between a local node and mining hardware using the Stratum v1 protocol and the DATUM Protocol for pool communication, ensuring the miner controls template creation.

Tokens
3.6K
Snippets
9
Records
13
Agent score
50%

What's inside DATUM Gateway

  1. What is the DATUM Gateway and how does it work?

    master

    The DATUM Gateway is a client-side tool for decentralized block template creation, enabling true solo mining. It acts as a bridge between a local Bitcoin node and mining hardware.

    Core Workflow:

    1. Fetch Templates: It reaches out to a local Bitcoin node via RPC using the GBT (getblocktemplate) mechanism.
    2. Generate Work: It generates and distributes mining work to hardware using the Stratum v1 protocol (with ASICBoost extensions).
    3. Submit Solutions: It submits solved blocks directly to the network.
    4. Pool Communication: For miners wanting pooled rewards, it uses the DATUM Protocol to communicate with a DATUM-supporting pool (like DATUM Prime). The pool coordinates reward splits but does not create the work; the work is always generated from the miner's local node.

    Key Protocols Supported:

    • Mining Hardware: Stratum v1 with version rolling extensions (ASICBoost).
    • Bitcoin Node: RPC via GBT (getblocktemplate).
    • Mining Pool: DATUM Protocol (encrypted and obfuscated link between Gateway and pool).
  2. Use worker names in Stratum usernames

    master

    To specify a worker name, append a period (.) followed by the name to your Bitcoin address in the miner's Stratum username configuration.

    Format: [bitcoin_address].[worker_name]

    Special Case: If the Stratum username begins with a period (e.g., .my_worker), the Gateway interprets it as a worker name only and appends it to the default mining.pool_address before sending it to the pool.

    Limitations:

    • The DATUM Gateway has a limit of 191 characters for Stratum usernames.
    • Individual miners may have much lower limits (e.g., Avalons at 63 characters, Whatsminer at 127 characters).
    bitcoin_address.workername
  3. Run the DATUM Gateway container

    master

    You can run the DATUM Gateway container using default settings or by mounting a custom configuration file. The container expects a configuration file at /app/config/config.json.

    # Run with default configuration
    docker run -p 23334:23334 -p 7152:7152 --name datum-gateway datum_gateway
    
    # Run with a custom configuration directory
    docker run -v /path/to/your/config/directory:/app/config -p 23334:23334 -p 7152:7152 datum_gateway
  4. Configure revenue sharing with username modifiers

    master

    The Gateway allows you to distribute shares to different addresses using "username modifiers". This feature works by reassigning share submissions based on the proof-of-work hash.

    Requirements:

    • This feature only works if datum.pool_pass_full_users is set to true.
    • Modifiers must be defined in the stratum section of your configuration as a JSON object.
    • The sum of proportions for a modifier must equal exactly 100% (1.0).

    How to use a modifier: Append a tilde (~) and the modifier name to your Stratum username: [bitcoin_address].[worker_name]~[modifier_name].

    Example Configuration:

    {
      "stratum": {
        "username_modifiers": {
          "modifier_name_1": {
            "bitcoin_address_A": 0.2,
            "": 0.8
          }
        }
      }
    }

    In this example, 20% of shares are redirected to bitcoin_address_A, and 80% are sent to the address specified in the Stratum username (represented by the empty string ""). The worker name is preserved and copied to all redirected addresses.

    {
      "stratum": {
        "username_modifiers": {
            "modifier name 1": {
                "bitcoin address A": 0.2,
                "": 0.8
            }
        }
      }
    }
  5. Configure Bitcoin Node connection for Docker

    master

    When running DATUM Gateway in Docker, you must configure the rpc_host, rpc_port, rpc_user, and rpc_pass in your config.json. Additionally, you must disable the notify fallback in your configuration file.

    In your Bitcoin node's bitcoin.conf, you must set the blocknotify parameter to call the DATUM Gateway's notify endpoint.

    ### 1. Bitcoin Node in the same Docker network
    
    **config.json**
    ```json
    {
      "rpc_host": "bitcoin-node",
      "rpc_port": 8332,
      "rpc_user": "your_rpc_user",
      "rpc_pass": "your_rpc_password"
    }

    bitcoin.conf

    blocknotify=wget -q -O /dev/null http://datum-gateway:7152/NOTIFY

    2. Bitcoin Node on the Host System

    Option A: Using host.docker.internal (Recommended)

    config.json

    {
      "rpc_host": "host.docker.internal",
      "rpc_port": 8332,
      "rpc_user": "your_rpc_user",
      "rpc_pass": "your_rpc_password"
    }

    Option B: Using host networking mode

    Run the container with --network host:

    docker run --network host -v /path/to/config:/app/config datum_gateway

    config.json

    {
      "rpc_host": "localhost",
      "rpc_port": 8332,
      "rpc_user": "your_rpc_user",
      "rpc_pass": "your_rpc_password"
    }

    bitcoin.conf

    blocknotify=wget -q -O /dev/null http://localhost:7152/NOTIFY

    3. Bitcoin Node on a Remote System

    config.json

    {
      "rpc_host": "192.168.1.100",
      "rpc_port": 8332,
      "rpc_user": "your_rpc_user",
      "rpc_pass": "your_rpc_password"
    }

    bitcoin.conf

    blocknotify=wget -q -O /dev/null http://datum-gateway-host-ip:7152/NOTIFY
  6. Install DATUM Gateway

    master

    Prerequisites

    • 64-bit AMD or Intel system.
    • Linux-based operating system.
    • A fully synced Bitcoin full node (Bitcoin Knots is highly recommended for better template control).
    • Required libraries: libcurl, libjansson, libmicrohttpd, libsodium.
    • Hardware: Mining hardware must be able to reach the system running the Gateway.
    • RAM: ~1GB base + 1GB per 1000 Stratum clients + Bitcoin node requirements.

    Step 1: Install Dependencies

    Install the required development packages based on your OS:

    Debian/Ubuntu:

    sudo apt install cmake pkgconf libcurl4-openssl-dev libjansson-dev libsodium-dev libmicrohttpd-dev psmisc

    Fedora/Amazon Linux:

    sudo dnf install cmake pkgconf libcurl-devel jansson-devel libsodium-devel libmicrohttpd-devel psmisc

    Alma Linux:

    sudo dnf install epel-release dnf-plugins-core
    sudo dnf config-manager --set-enabled crb
    sudo dnf install cmake pkgconf libcurl-devel jansson-devel libsodium-devel libmicrohttpd-devel psmisc

    Oracle Linux:

    sudo dnf install epel-release dnf-plugins-core
    sudo dnf config-manager --set-enabled ol9_codeready_builder
    sudo dnf install cmake pkgconf libcurl-devel jansson-devel libsodium-devel libmicrohttpd-devel psmisc

    Alpine:

    sudo apk add build-base cmake pkgconf argp-standalone curl-dev jansson-dev libsodium-dev libmicrohttpd-dev psmisc

    Arch:

    sudo pacman -Syu base-devel cmake pkgconf curl jansson libsodium libmicrohttpd psmisc

    Clear Linux:

    sudo swupd bundle-add c-basic cmake pkgconf devpkg-curl devpkg-jansson devpkg-libsodium devpkg-libmicrohttpd psmisc

    FreeBSD:

    sudo pkg install cmake pkgconf curl jansson libsodium libmicrohttpd argp-standalone libepoll-shim

    Step 2: Compile

    From the source directory, run:

    cmake . && make
  7. Run and Configure DATUM Gateway

    master

    After compilation, follow these steps to run the Gateway:

    1. View Options: Run the executable with the -? flag to see detailed configuration information, descriptions, and required options.
      ./datum_gateway -?
    2. Create Configuration: Construct a configuration file. By default, it looks for datum_gateway_config.json in the current working directory.
      • Use doc/example_datum_gateway_config.json as a template.
      • Security: Set a strong API/web admin password. This password is also used to prevent CSRF attacks on the web interface.
      • Coinbase Tags: Set your coinbase tags. While the primary tag is unused in pooled mining, the secondary tag is used by block explorers to identify your mined blocks.
    3. Start Mining: Once configured, point your mining hardware at the Gateway's address.
    # View help and required options
    ./datum_gateway -?
  8. Configure your Bitcoin node for DATUM Gateway

    master

    To use the DATUM Gateway, your Bitcoin node must be configured to support GBT and reserve sufficient space for the pool's generation transaction.

    Recommended Node Settings:

    • Block Size/Weight: You must reserve space for the generation transaction. Currently recommended:
      • blockmaxsize=3985000
      • blockmaxweight=3985000
    • Mempool & Reconstruction: To maximize valid transactions:
      • maxmempool=1000
      • blockreconstructionextratxn=1000000

    Handling Block Notifications: To avoid mining stale work, the Gateway needs to know when new blocks arrive.

    • If Gateway and Node are on the same system: Run the Gateway as the same user as the full node and add this to your bitcoin.conf: blocknotify=killall -USR1 datum_gateway (requires psmisc package).
    • If on different systems: Use the NOTIFY endpoint on the Gateway's dashboard/API instead.

    RPC Access: Ensure the Gateway has RPC access to your node. You must add an RPC user to your node configuration and whitelist the Gateway's IP/service for RPC access.

    # bitcoin.conf recommendations
    blockmaxsize=3985000
    blockmaxweight=3985000
    maxmempool=1000
    blockreconstructionextratxn=1000000
    blocknotify=killall -USR1 datum_gateway
  9. Configure the default Bitcoin address

    master

    The DATUM Gateway requires a valid Bitcoin address to start. This address is used as the default username for all mining operations. You must set this using the mining.pool_address configuration option.

    Supported address formats:

    • Base58 (Legacy)
    • Bech32 (Segwit)
    • Bech32m (Taproot)

    Note: The Gateway supports both Bitcoin and Bitcoin testnet, but it will not detect if you provide an address for the wrong network.

    {
      "mining": {
        "pool_address": "YOUR_BITCOIN_ADDRESS"
      }
    }
  10. Manage how Stratum usernames are passed to the pool

    master

    You can control how the Gateway handles the Stratum username (the username provided by your miner) when communicating with the pool using two configuration options in the datum section:

    1. Pass full username: Set datum.pool_pass_full_users to true. The Stratum username is passed to the pool exactly as provided by the miner.
    2. Pass as worker names: Set datum.pool_pass_full_users to false and datum.pool_pass_workers to true. The entire Stratum username is appended to the default mining.pool_address as a worker name.
    3. Ignore Stratum username: Set both datum.pool_pass_full_users and datum.pool_pass_workers to false. Only the mining.pool_address is used; the miner's username is ignored.

    Warning: Always test your full mining stack configuration. Misconfiguration can result in lost work that is impossible to recover.

    {
      "datum": {
        "pool_pass_full_users": true,
        "pool_pass_workers": false
      }
    }
  11. Known limitations and best practices

    master

    Connection Failures

    If the connection with the pool is lost and fails to reconnect, the Gateway will disconnect all stratum clients. This is intended to allow miners to use built-in failover to switch to non-DATUM mining or an alternate Gateway.

    Share Count Discrepancies

    Accepted/rejected share counts on mining hardware may not perfectly match the pool. This delta occurs because shares are validated against the local node first, and then again against pool requirements, latency to the pool (stale work), and latency between the node and the network (stale work). Stratum v1 does not support reporting previously accepted work as rejected to the miner.

    Best Practices

    • Failover: Always configure failover settings on your miners. When mining on a DATUM pool, set the miner's failover to use that pool's Stratum endpoint.
    • BETA Warning: This is a public BETA release. Rapid development and protocol changes may occur, requiring frequent updates to maintain compatibility.