pyubx2 UBX Protocol Parser and Generator

repository·master·Indexed 42 days ago

https://github.com/semuconsulting/pyubx2

A Python 3 parser and generator for the u-blox UBX protocol, as well as NMEA 0183 and RTCM3 protocols. It provides tools to read, parse, and generate GNSS messages for u-blox receiver modules, including support for the Generation 9 configuration interface (CFG-VALSET, CFG-VALDEL, CFG-VALGET) and geodetic calculation utilities.

Tokens
9.4K
Snippets
32
Records
43
Agent score
54%

What's inside pyubx2

  1. Understand the pyubx2 package structure

    master

    The pyubx2 package is organized into several specialized submodules for handling UBX (u-blox) messages. Developers can interact with different layers of the protocol through these modules:

    • Core Message Handling: pyubx2.ubxmessage for message objects and pyubx2.ubxreader for streaming/reading UBX data.
    • Type Definitions and Configuration: A suite of pyubxtypes_* modules (including _core, _configdb, _decodes, _get, _poll, and _set) that manage the underlying UBX data types and configuration interfaces.
    • Utilities and Exceptions: pyubx2.ubxhelpers for helper functions and pyubx2.exceptions for protocol-specific error handling.
    • Variants: pyubx2.ubxvariants for handling different UBX message variations.
  2. Use the UBX Configuration Interface (Generation 9+)

    master

    For Generation 9 UBX protocol devices (e.g., NEO-M9N, ZED-F9P), use the dedicated configuration interface (CFG-VALSET, CFG-VALDEL, CFG-VALGET) instead of legacy configuration messages. These messages allow setting, deleting, or polling parameters using keys (strings or hex IDs) in specific memory layers.

    Key Concepts:

    • Layers: SET_LAYER_RAM (1), SET_LAYER_BBR (2), SET_LAYER_FLASH (4).
    • Transactions: TXN_NONE (0), TXN_START (1), TXN_ONGOING (2), TXN_COMMIT (3).
    • Key Conversion: Use cfgname2key() to convert a string name to a hex ID, and cfgkey2name() for the reverse.
  3. Extend the UBX protocol definitions

    master

    The UBX protocol is defined via dictionaries in ubxtypes_*.py modules. To add support for a new message, you must add an appropriate dictionary entry to the relevant module. If a message has multiple payload variants for the same class, ID, and mode, it must be handled in ubxvariants.py.

    Payload Definition Rules:

    1. Uniqueness: Attribute names must be unique within a message class.
    2. Types: Attribute types must be valid UBX types (e.g., I1, U2, X4).
    3. Scaling: For scaled attributes, use a list containing the type and the scaling factor: {"attr_name": ["TYPE", factor]} (e.g., {"lat": ["I4", 1e-7]}).
    4. Repeating/Bitfield Groups: Suffix names with _grp or _bit and define them as a tuple ('numr', {dict}):
      • numr can be an integer (fixed count), a string (name of a preceding attribute), an 'X' type (bit flags), or None (variable size, must be at the end of the payload).
      • {dict} is the nested dictionary for the group items.

    Repeating attribute names are parsed using a two-digit suffix (e.g., svid_01, svid_02).

  4. Access attributes and repeating groups in UBXMessage

    master

    A UBXMessage object provides access to the parsed fields of a message via attributes.

    • Identity: Use msg.identity to get the message name (e.g., 'NAV-POSLLH').
    • Payload: Use msg.payload to get the raw bytes.
    • Repeating Groups: Attributes within repeating groups are named with a two-digit suffix (e.g., svid_01, svid_02).

    To iterate through a repeating group, use getattr with the group size (e.g., msg.numSV) to construct the attribute name dynamically.

    vals = []
    # Example: iterating through a repeating group of 'svid' attributes
    for i in range(msg.numSV):
        svid = getattr(msg, f"svid_{i+1:02d}")
        vals.append(svid)
    print(vals)
  5. How UBX message categories (GET, SET, POLL) work

    master

    The UBX protocol distinguishes between different message directions and purposes using the mode or msgmode parameter:

    • GET (0x00): Output from the receiver (default). Used for streaming/parsing receiver data.
    • SET (0x01): Command input to the receiver. Used for configuring the device.
    • POLL (0x02): Query input to the receiver. Used to request specific data.
    • SETPOLL (0x03): Automatically determines whether the message is SET or POLL based on the payload.

    If you are simply parsing output from a receiver, the mode is implicitly GET. If you are creating or parsing input messages (commands/queries), you must explicitly set the mode to SET or POLL.

  6. Explore pyubx2 usage examples

    master

    The repository contains several example scripts in the \examples folder that demonstrate common patterns:

    • ubxoptions.py: Parsing and constructing UBX messages.
    • ubxpoller.py: Concurrent reading/writing/displaying using threads and queues.
    • ubxsetrates.py: Using legacy CFG-MSG to set navigation rates.
    • ubxconfigdb.py: Using Generation 9 configuration interface (CFG-VALSET, CFG-VALDEL, CFG-VALGET).
    • ubxfactoryreset.py: Sending CFG-CFG factory reset commands.
    • ubxfile.py: Implementing a binary file reader using UBXReader iterators.
    • ubxsocket.py: Implementing a TCP Socket reader using UBXReader iterators.
    • gpxtracker.py: Converting UBX binary dumps to *.gpx files.
    • ubxserver.py: A web server wrapper around pyubx2.UBXreader providing a web page or RESTful API.
    • mon_span_spectrum.py: Plotting spectrum analysis from MON-SPAN messages using matplotlib.
    • utilities.py: Various pyubx2 utility methods.
  7. Install pyubx2 via pip or conda

    master

    Install the latest version of pyubx2 using pip. The library is compatible with Python >= 3.10.

    To install into a virtual environment:

    python3 -m venv env
    source env/bin/activate # (or env\Scripts\activate on Windows)
    python3 -m pip install --upgrade pyubx2

    For Conda users, install from conda-forge:

    conda install -c conda-forge pyubx2
    python3 -m pip install --upgrade pyubx2
  8. Handle UBX message variants with multiple payload definitions

    master

    Some UBX messages share the same Class ID and Message ID but have different payload structures depending on the specific variant (e.g., version, type, or length). The pyubx2.ubxvariants module provides helper functions to resolve the correct payload dictionary for these messages.

    These functions typically inspect kwargs for specific indicators like version, type, tpIdx, datumNum, or the payload itself to determine which specific UBX payload definition should be used for parsing or generating the message.

  9. Troubleshoot common UBX errors

    master

    Unknown Protocol

    Usually caused by serial data corruption or incorrect configuration.

    • Verify UART1/UART2 baud rates (factory default is 38400, not 9600).
    • Ensure no other process (like gpsd) is using the serial port.

    Serial Permission

    • Linux: Ensure your user is a member of the tty and/or dialout groups.
    • Ensure no other process is contending for the port.

    UnicodeDecode

    • When reading UBX data from a log file, ensure you open the file in read binary mode: stream = open('ubxdatalog.log', 'rb')

    Spurious u-center data (CFG-VALGET, DBG, etc.)

    If you see undocumented message classes in recordings made with u-center:

    • When recording in u-center, select 'No' when prompted to 'Add Receiver Configuration'.
    • Avoid using the 'debug' option in u-center.
  10. Generate UBX messages with UBXMessage

    master

    You can create a UBXMessage object using its constructor. This allows you to manually construct any UBX message by specifying its class, ID, and payload.

    Constructor Signature: pyubx2.ubxmessage.UBXMessage(ubxClass, ubxID, mode: int, **kwargs)

    Parameters:

    • ubxClass: Message class (can be a lookup string, integer, or bytes from pyubx2.UBX_CLASSES).
    • ubxID: Message ID (can be a lookup string, integer, or bytes from pyubx2.UBX_MSGIDS).
    • mode: Operation mode (0=GET, 1=SET, 2=POLL).
    • **kwargs: Optional keyword arguments representing the message payload.
      • payload: (Optional) A single sequence of bytes representing the full payload. Required for messages with 'variable by size' repeating groups.
      • Individual attributes: One or more keyword arguments corresponding to message attributes. Unspecified attributes are set to nominal values.
      • parsebitfield: (Optional) 1 to define bitfields as individual bits (default), 0 to define as byte sequences.

    Payload Definition Methods:

    1. Full Bytes: Pass payload=b'...'. Use this for variable-sized messages.
    2. Individual Attributes: Pass attributes as keyword arguments (e.g., rateUART1=1).
    3. Minimal: If no keyword arguments are passed, the payload is assumed to be null.
    from pyubx2 import UBXMessage, SET
    
    # Method A: Pass entire payload as bytes
    msg1 = UBXMessage(b'\x06', b'\x01', SET, payload=b'\x01\x03\x00\x01\x00\x00\x00\x00')
    
    # Method B: Pass individual attributes as keyword arguments
    msg2 = UBXMessage(0x06, 0x01, SET, msgClass=0x01, msgID=0x03, rateDDC=0, rateUART1=1, rateUART2=0, rateUSB=0, rateSPI=0)
    
    # Method C: Pass selected attributes (others default to nominal/0)
    msg3 = UBXMessage('CFG','CFG-MSG', SET, msgClass=0x01, msgID=0x03, rateUART1=1)
  11. Delete configuration parameters with UBXMessage.config_del()

    master

    Use UBXMessage.config_del() to unset (delete) up to 64 parameter settings in non-volatile memory layers.

    Parameters:

    • layers: Memory layer (e.g., SET_LAYER_FLASH).
    • transaction: Transaction mode (e.g., TXN_NONE).
    • keys: An array of up to 64 keys (either int keyID or str keyname).
    from pyubx2 import UBXMessage, SET_LAYER_FLASH, TXN_NONE
    
    layers = SET_LAYER_FLASH
    transaction = TXN_NONE
    keys = ["CFG_UART1_BAUDRATE", 0x40520001]
    msg = UBXMessage.config_del(layers, transaction, keys)
    # msg.serialize() can then be used to send the command
  12. Parse individual UBX messages with UBXReader.parse()

    master

    If you have a standalone bytes array containing a UBX message, use the static UBXReader.parse(data) method to convert it into a UBXMessage object.

    Note: UBXMessage objects are immutable once instantiated.

    Arguments

    • validate: VALCKSUM (0x01, default) or VALNONE (0x00).
    • parsebitfield: 0 (bytes), 1 (flags), or 2 (both).
    • msgmode: GET (0), SET (1), POLL (2), or SETPOLL (3).
    from pyubx2 import UBXReader
    
    # Parsing a GET message
    msg = UBXReader.parse(b'\xb5b\x05\x01\x02\x00\x06\x01\x0f\x38')
    
    # Parsing a SET message
    msg_set = UBXReader.parse(b'\xb5b\x13\x40\x14\x00\x01\x00\x01\x02\x01\x02\x03\x04\x01\x02\x03\x04\x01\x02\x03\x04\x01\x02\x03\x04\x93\xc8', msgmode=1)