microReticulum

repository·master·Indexed 43 days ago

https://github.com/attermann/microreticulum

A C++ port of the Reticulum Network Stack (RNS) optimized for 32-bit microcontrollers. It provides the core networking logic for implementing Reticulum Transport Nodes, featuring support for UDP and LoRa transports, a Bitcask-style storage engine via microStore, and a schema-driven Provisioning subsystem using MsgPack for device configuration.

Tokens
31.5K
Snippets
68
Records
115
Agent score
58%

What's inside microReticulum

  1. Overview of available example applications

    master

    The repository provides several sample programs to demonstrate microReticulum usage, categorized by build system and transport type.

    Native Examples (CMake)

    • udp_transport: UDP-based transport node.
    • udp_announce: UDP announce example.
    • link_native: Link establishment over a native interface.
    • nomadnet: NomadNet-style application example.

    Note: The CMake build also produces a shared helper library udp_interface located in examples/common/udp_interface/ used by UDP examples and interop senders.

    LoRa Examples (PlatformIO)

    • lora_transport/: LoRa radio transport node.
    • lora_announce/: LoRa announce example.
    • lora_transport_internalfs_override/: LoRa transport with InternalFS persistence override.
  2. Understand microReticulum Provisioning transport assumptions

    master

    When building a client for the microReticulum Provisioning subsystem, keep the following transport constraints in mind:

    • MsgPack Framing: Every request and response is a single, complete MsgPack-encoded Bytes blob. The library does not support chunking or streaming; you must receive the full response before proceeding.
    • Synchronous Execution: The firmware-side implementation is synchronous and reentrant-unsafe. Clients must serialize requests; do not attempt to have multiple parallel in-flight operations to the same device.
    • Endianness: The protocol is little-endian-agnostic because MsgPack handles multi-byte encoding.
    • Security: There is no built-in authentication or encryption at this layer. Security must be implemented in the transport layer (e.g., TLS, encrypted BLE, etc.) that wraps these MsgPack frames.
  3. Handle Schema Versioning in clients

    master

    The GET_INFO operation returns a SchemaVersion key. Clients should check this version to determine how to parse the GET_SCHEMA response layout.

    Current Version: 2

    VersionNamespace Entry Format
    1[id, name, [fields]]
    2[id, name, parent_id, [fields]] (where parent_id == 0 is root)

    Compatibility Tip: A v1-compatible client can still process v2 responses by reading only the first three elements of the namespace array (id, name, and fields), effectively treating the schema as a flat list.

  4. Understand flash persistence for namespaces

    master

    When RNS_USE_FS is enabled, namespaces are persisted as MsgPack files.

    Persistence Details

    • File Path: Namespaces use a dotted path based on their hierarchy (e.g., Interfaces.LoRa.msgpack). Flat namespaces are backward compatible.
    • Format: A MsgPack map where keys are field IDs (uint) and values are the typed data.
    • Atomicity: Writes are atomic. Data is written to a .tmp file first, then rename()'d over the live file to prevent corruption during power loss.
    • Forward Compatibility: Unknown field IDs in a file are silently ignored during loading.
    • Exclusions: FF_READ_ONLY and FF_WRITE_ONLY fields are not persisted.
    • Security: FF_SECRET fields are persisted to disk. The Provisioning subsystem does not provide encryption-at-rest; users must implement an outer encryption layer if storing sensitive material.
  5. Understand Provisioning working, draft, and commit semantics

    master

    Each namespace manages two internal maps: Working (the current effective value) and Draft (pending edits).

    State Operations

    OperationEffect
    Provisioner::field(ns, f, Source::Working)Read: getter if present, else working map.
    Provisioner::field(ns, f, Source::Draft)Returns the draft value if any, else Value::None.
    Provisioner::field(ns, f, Source::Effective)Draft if present, else getter/working.
    Provisioner::field(ns, f, value) (set)Validates type/constraints, writes to draft. Rejects FF_READ_ONLY.
    Provisioner::commit(ns)Promotes drafts $\rightarrow$ working, fires setters, persists to flash.
    Provisioner::discard(ns)Drops drafts; working remains unchanged.
    Provisioner::factory_reset()Resets all working to defaults, fires setters with defaults, removes namespace files.

    Note: These operations are driven by the wire commands SET_STATE, COMMIT, DISCARD, and FACTORY_RESET.

  6. Verify wire-format compatibility with Python Reticulum

    master

    The C++ implementation uses wire-format cross-validation to ensure compatibility with Python Reticulum (e.g., version 1.2.9).

    1. Byte-level validation: The C++ unit test test/test_resource_advertisement uses a PYTHON_GOLDEN_HEX byte sequence captured from Python's RNS.Resource.ResourceAdvertisement.pack(). It asserts that ResourceAdvertisement::unpack in C++ correctly parses these bytes and that ResourceAdvertisement::pack in C++ produces the exact same byte sequence.
    2. Regenerating golden bytes: If the Python reference's wire format changes, use encode_advertisement.py to regenerate the golden bytes so the C++ test constants can be updated.
  7. Format the Provisioning Request/Response envelope

    master

    All communication follows a strict MsgPack array format consisting of exactly 3 elements:

    [ op_id, seq, payload ]

    PositionTypeDescription
    0uint8Operation ID: Determines the action (see Operation IDs).
    1uint64Sequence Number: A caller-supplied ID used to match responses to requests. Use 0 if matching is not required.
    2variesPayload: The operation-specific data, or nil if the operation has no input or body.

    Error Handling Note: In a successful response, the op_id matches the request. If an error occurs, the op_id in the response will be 101 (ERROR). You can recover the original operation ID by checking the seq field in the error response.

    // Example Request: [op_id, seq, payload]
    [1, 12345, null]
    
    // Example Response: [op_id, seq, payload]
    [1, 12345, {"schema": ...}]
  8. Optimize GET_STATE with PriorHash caching

    master

    To reduce bandwidth, clients can implement a short-circuit cache using the Hash field returned in GET_STATE responses. The hash is a CRC32 of the response body (excluding the hash itself).

    How to use it:

    1. On a GET_STATE request, include the previously received hash in the PriorHash field of the request envelope.
    2. The hash must use the same scope (the same NamespaceFilter and Draft setting).
    3. If the server's current state matches the hash, it returns { Unchanged: true } instead of the full payload.

    Cache Key Requirements: A cache key must be a combination of (NamespaceFilter, Draft). A hash for Draft: false is not valid for Draft: true.

    Invalidation: Discard cached hashes after any SET_STATE, COMMIT, DISCARD, FACTORY_RESET, or REBOOT operations.

    # Example logic for cache key
    cache_key = (namespace_filter, is_draft)
  9. Define metrics and commands in a namespace

    master

    Provisioning supports two specialized field types for non-persistent data:

    1. Metrics (metric_*): Read-only values used for instrumentation (e.g., packet counts, RSSI). The provided getter is consulted every time a client requests the value. These do not appear in GET_STATE as writable fields.
    2. Commands (command_*): One-shot, write-only actions (e.g., ping, reboot_in_seconds). The setter executes the action immediately using the provided argument. Commands are not persisted and do not appear in GET_STATE.
    // Metrics Example
    Provisioner::instance().register_namespace("Stats", NS_STATS)
        .metric_int("packets_received",  1, []() { return (int64_t)Transport::packets_received(); })
        .metric_float("rssi_dbm",        3, []() { return radio.last_rssi(); })
        .end();
    
    // Commands Example
    Provisioner::instance().register_namespace("Actions", NS_ACTIONS)
        .command_bool("ping", 1, [](const Value&) { send_test_packet(); return true; })
        .command_int("reboot_in_seconds", 2, 0, 3600, [](const Value& v) { schedule_reboot(v.as_int()); return true; })
        .end();
  10. Maintain ID stability for namespaces and fields

    master

    The Provisioning system is schema-driven and relies on stable IDs. To prevent silent failures when reading from flash, follow these rules:

    1. Never reuse an ID: Once a namespace or field ID is used, it is permanently assigned. If a field is removed, reserve its ID in a comment so it is never reused.
    2. Never re-assign meaning: If a field's semantics change (e.g., changing units or type), allocate a new ID and retire the old one.
    3. Renames are safe: The human-readable name is purely informational; only the ID matters.
    4. Type widening is OK: Increasing the range of an INT field is acceptable, but changing the type (e.g., INT to STRING) requires a new ID.
    RangeOwner
    Namespace ID 0Reserved (engine). Do not use.
    Namespace IDs 1-99Core library (Reticulum=1, Transport=2; ID 3 reserved).
    Namespace IDs 100-199Official microReticulum companion apps.
    Namespace IDs 200-65535Third-party / vendor / per-installation.
    Field ID 0Reserved (schema version) within every namespace.
    Field IDs 1-65535Namespace-owned; allocate append-only.

    Implementation Pattern

    Use a single source of truth header to manage IDs and track the next available ID using a comment.

    // my_app/IdsForProvisioning.h
    namespace MyApp::Ns {
        constexpr uint16_t Id = 200;
        namespace Field {
            constexpr uint16_t SchemaVersion = 0;       // reserved
            constexpr uint16_t DeviceName    = 1;
            constexpr uint16_t Brightness    = 2;
            // constexpr uint16_t _Reserved3 = 3;        // (was Heading, removed v0.4)
            constexpr uint16_t Volume        = 4;
            // next-id: 5
        }
    }
  11. Use namespace-level commit hooks for cross-field validation

    master

    A namespace can register an optional commit hook that fires once per COMMIT pass, before any field setters run. This is the ideal place to validate combinations of related fields or veto inconsistent change sets.

    Hook Lifecycle and Capabilities

    • Timing: Fires after drafts are received but before any field setters are executed. This allows you to inspect or revert drafts before they affect the runtime.
    • Execution: Only fires if the namespace has at least one pending draft.
    • Operations via Namespace&:
      • ns.has_draft(fid) / ns.draft(fid, out): Inspect pending changes.
      • ns.effective(fid): Get the value that will be used (the draft if present, otherwise the current working value).
      • ns.clear_draft(fid): Revert a specific field's change.
      • ns.clear_draft(): Revert all pending changes in the namespace (effectively a no-op commit).
      • ns.set_draft(fid, v): Amend a draft with a new value (triggers the field's validator).

    When to use a Hook vs a Setter

    • Use a Setter: For constraints affecting a single field in isolation.
    • Use a Hook: For multi-field validation, enforcing field ordering, or implementing "all-or-nothing" commit semantics.
    Provisioner::instance()
        .register_namespace("radio", 100)
            .field_int  ("sf",        1, FF_REBOOT_REQUIRED, 8, 7, 12)
            .field_float("bandwidth", 2, FF_REBOOT_REQUIRED, 125e3, 7.8e3, 500e3)
            .on_commit([](Namespace& ns) {
                // Cross-field validation: SF12 + BW7.8k is allowed but SF12 + BW500k
                // saturates the chip. Revert just the bandwidth draft if the
                // combination is illegal.
                Value sf, bw;
                const bool has_sf = ns.draft(1, sf);
                const bool has_bw = ns.draft(2, bw);
                const int64_t eff_sf = has_sf ? sf.as_int() : ns.effective(1).as_int();
                const double  eff_bw = has_bw ? bw.as_float() : ns.effective(2).as_float();
                if (eff_sf >= 11 && eff_bw > 250e3) {
                    ns.clear_draft(2);   // bandwidth change refused
                }
            })
        .end();
  12. Build native examples using CMake

    master

    Native examples (UDP transport, UDP announce, native link establishment, and NomadNet-style applications) can be built using CMake on Linux, macOS, or Raspberry Pi. The CMake target names correspond to the directory names.

    To build all native examples at once, use the --target flag with the specific target names. You can also skip building examples entirely during the configuration step if you are building the parent project.

    # Configure the build directory
    cmake -S . -B build
    
    # Build all native examples
    cmake --build build -j --target udp_transport udp_announce link_native nomadnet
    
    # Build a single example
    cmake --build build -j --target udp_transport
    
    # Skip building examples during configuration
    cmake -S . -B build -DRNS_BUILD_EXAMPLES=OFF