pyais: AIS Message Decoding

repository·main·Indexed 42 days ago

https://github.com/m0r13n/pyais

A pure Python library for encoding and decoding Automatic Identification System (AIS) messages. It supports AIVDM/AIVDO formats and handles single messages, files, and TCP/UDP streams. Features include a filtering system via FilterChain, vessel tracking with AISTracker, support for Gatehouse wrappers, and a CLI tool called ais-encode for generating NMEA AIS messages from JSON.

Tokens
30.6K
Snippets
118
Records
140
Agent score
53%

What's inside pyais

  1. How to encode AIS messages correctly

    main

    When creating AIS messages for encoding, you should always use the .create() class method (e.g., MessageType1.create(...)) instead of instantiating the class directly. The .create() method automatically sets default values for missing attributes.

    Important Constraints:

    • MessageTypeX.create(...) requires at least the mmsi keyword argument.
    • If you instantiate a message class directly (e.g., MessageType1(mmsi=1, ...)), you must provide every single possible attribute for that message type, otherwise a TypeError will be raised.
  2. How AIS filters and FilterChain work together

    main

    The AIS filtering system uses a modular approach where individual filter classes evaluate messages based on specific criteria. These filters are orchestrated by a FilterChain object, which processes an AIS message stream sequentially.

    The Workflow:

    1. AIS Stream: A stream of AIS messages is provided to the FilterChain.
    2. Sequential Application: The FilterChain passes messages through each filter in the order they were provided during initialization.
    3. Passing Criteria: Each filter evaluates the message; if the message meets the filter's criteria, it is passed to the next filter in the chain. If it fails, it is dropped from the stream.

    This allows you to build complex logic by combining simple, specialized filters.

    from pyais import FilterChain
    from pyais.filters import AttributeFilter, MessageTypeFilter
    
    # Create a chain that first checks a custom attribute, then checks message type
    chain = FilterChain([
        AttributeFilter(lambda x: hasattr(x, 'turn')),
        MessageTypeFilter(1, 2, 3)
    ])
    
    # Use the chain to filter a stream
    filtered_data = chain.filter(stream)
  3. Access Gatehouse wrappers in streaming APIs

    main
    Some AIS messages are encapsulated in Gatehouse wrappers (e.g., $PGHP...). These are parsed by pyais only when using streaming classes such as FileReaderStream or TCPStream. The wrapper information can be accessed via the .wrapper_msg attribute on every NMEASentence object. This attribute is None by default if no wrapper is present.
  4. Handling binary payloads in AIS messages

    main

    While pyais supports all AIS message types, it does not decode message-specific binary payloads for types 6, 8, 25, 26, and 27. These messages contain unspecified binary data that must be interpreted by the user based on the dac (Designated Area Code) and fid (Functional ID).

    You can access the raw payload as bytes via the .data attribute. To assist with manual decoding, pyais provides utility functions to convert between bits and bytes.

    # Accessing raw payload from a message of type 8
    msg = NMEAMessage(b"!AIVDM,1,1,,A,85Mwp`1Kf3aCnsNvBWLi=wQuNhA5t43N`5nCuI=p<IBfVqnMgPGs,0*47").decode()
    payload = msg.data
    
    # Using utilities to transform data
    from pyais.utils import bits2bytes, bytes2bits
    
    bits2bytes('00100110')      #=> b'&'
    bytes2bits(b'&').to01()     #=> '00100110'
  5. How AIS decoding works in pyais

    main

    Decoding in pyais follows a three-step process:

    1. NMEA 0183 Parsing: The raw input is parsed at the physical protocol layer into a NMEASentence. This layer handles the outer protocol used for data transmission.
    2. AIS Sentence Parsing: The NMEASentence is parsed into an AISSentence. This layer extracts AIS-specific metadata like the AIS message ID, fragment count/number, sequence number, and the raw AIS payload.
    3. Payload Decoding: The AISSentence payload is finally decoded into a high-level AISMessage object based on its specific AIS ID (there are 27 different top-level message types).
  6. How decoding handles missing bits

    main
    When decoding an AIS message, if the payload contains fewer bits than are required to populate every field defined in the message class, the remaining fields will be set to None rather than their default values.
  7. Use NMEAQueue to assemble NMEA sentences

    main

    The NMEAQueue class is used to manage and assemble NMEA sentences from raw byte streams. It handles both single-line sentences and multi-line messages by buffering fragments until a complete message is formed. It can also integrate with a TagBlockQueue to facilitate tag block processing.

    Key behaviors:

    • Multi-line Assembly: Buffers fragments based on sequence ID and channel, assembling them once all parts are present.
    • Error Resilience: Malformed or invalid sentences are automatically skipped without raising exceptions.
    • Tag Block Integration: If a TagBlockQueue is provided during initialization, sentences are passed to it for processing.
    from pyais.stream import TagBlockQueue
    from my_module import NMEAQueue
    
    # Initialize a TagBlockQueue
    tbq = TagBlockQueue()
    
    # Create an NMEAQueue instance
    queue = NMEAQueue(tbq=tbq)
    
    # Add a line of raw bytes to the queue
    queue.put_line(b"!AIVDM,1,1,,A,15N:;R0P00PD;88MD5NS8v2P00,0*3C")
    
    # Retrieve the assembled sentence
    sentence = queue.get_or_none()
    if sentence:
        print(sentence)
  8. Use TagBlockQueue for grouping NMEA sentences

    main

    When using streaming APIs (like FileReaderStream or TCPStream), you can provide an optional tbq keyword argument set to a TagBlockQueue(). This allows you to handle groups of NMEA sentences that belong together. You can use tbq.get_nowait() to retrieve a list of NMEASentence objects only when the entire group has been fully received.

    from pyais.stream import FileReaderStream
    from pyais.stream import TagBlockQueue
    import queue
    
    with FileReaderStream('/path/to/file.nmea', tbq=TagBlockQueue()) as stream:
        tbq = stream.tbq
    
        for msg in stream:
            try:
                # Retrieves the complete group of sentences
                print(tbq.get_nowait())
            except queue.Empty:
                pass
  9. Understand the NMEASentence abstraction

    main

    The NMEASentence class is the base abstraction for all NMEA parsing in pyais. While it is not intended to be instantiated directly (it is used by subclasses like GatehouseSentence and AISSentence), every instance of an NMEA sentence provides the following attributes:

    • raw: The raw decoded message as bytes.
    • delimiter: The start delimiter (typically $ or ?).
    • data_fields: The NMEA payload (interpretation depends on the sentence type).
    • talker_id: The two-letter talker ID.
    • type: The three-letter type code.
    • checksum: The 8-bit XOR checksum.
    • is_valid: Boolean indicating if the checksum is valid.
    • wrapper_msg: An optional encapsulating message (used for Gatehouse wrappers).
    msg = NMEASentence(b"!AIVDM,1,1,,A,15Mj23P000G?q7fK>g:o7@1:0L3S,0*1B")
    
    print(msg.raw)
    print(msg.data_fields)
    print(msg.is_valid)
  10. Run pyais using Docker

    main

    You can run your application inside a container by building the image locally and then running it in interactive mode. Once inside the container, you can execute provided examples.

    1. Build the image: docker build . -t pyais

    2. Run the container in bash: docker run -it --rm pyais /bin/bash

    3. Run an example (e.g., live stream): python ./examples/live_stream.py

    docker build . -t pyais
    docker run -it --rm pyais /bin/bash
    # Inside container:
    python ./examples/live_stream.py
  11. How to work with tag blocks

    main

    Tag blocks are optional key-value pairs wrapped between \s prefixes in NMEA sentences, used to hold extra information. In pyais, tag blocks are lazily decoded to save resources. To access the data, you must check if msg.tag_block is not None and explicitly call msg.tag_block.init() before accessing its contents via asdict().

    Supported tag block fields include:

    • c: Receiver timestamp in Unix epoch
    • d: Destination station
    • n: Line count
    • r: Relative time
    • s: Source station
    • t: Text

    Unknown fields are omitted.

    from pyais.stream import IterMessages
    
    text = """
    \s:2573135,c:1671620143*0B\!AIVDM,1,1,,A,16:=?;0P00`SstvFnFbeGH6L088h,0*44
    """
    
    messages = [line.encode() for line in text.split() if line]
    
    with IterMessages(messages) as s:
        for msg in s:
            if msg.tag_block is not None:
                # Must call .init() because tag blocks are lazily parsed
                msg.tag_block.init()
                print(msg.tag_block.asdict())
            print(msg.decode())
  12. Filter AIS messages using FilterChain

    main

    The filtering system allows you to chain multiple criteria to process an AIS stream. You create individual filter objects and pass them as a list to a FilterChain.

    Available filters:

    • AttributeFilter(func): Keeps messages where func(msg) is True.
    • NoneFilter(*attr_names): Filters out messages where the specified attributes are None.
    • MessageTypeFilter(*types): Keeps only specified message types.
    • DistanceFilter(ref_point, distance_km): Keeps messages within a certain distance (km) from a (lat, lon) tuple.
    • GridFilter(lat_min, lon_min, lat_max, lon_max): Keeps messages within a geographical bounding box.

    Use chain.filter(stream) to apply the filters to an AIS stream.

    from pyais import TCPConnection
    from pyais.filters import AttributeFilter, NoneFilter, MessageTypeFilter, DistanceFilter, GridFilter, FilterChain
    
    # Define filters
    attribute_filter = AttributeFilter(lambda x: not hasattr(x, 'turn') or x.turn == -128.0)
    none_filter = NoneFilter('lon', 'lat', 'mmsi2')
    message_type_filter = MessageTypeFilter(1, 2, 3)
    distance_filter = DistanceFilter((51.900, 5.320), distance_km=1000)
    grid_filter = GridFilter(lat_min=50, lon_min=0, lat_max=52, lon_max=5)
    
    # Create and apply the chain
    chain = FilterChain([
        attribute_filter,
        none_filter,
        message_type_filter,
        distance_filter,
        grid_filter,
    ])
    
    stream = TCPConnection(host='...', port=...)
    filtered_data = list(chain.filter(stream))
    
    for msg in filtered_data:
        print(msg.lat, msg.lon)