asn1tools

repository·master·Indexed 43 days ago

https://github.com/eerimoq/asn1tools

A Python package for parsing, encoding, and decoding ASN.1 specifications. It supports a wide variety of codecs including BER, DER, PER, UPER, XER, OER, GSER, and JER. The library provides a Specification class for mapping ASN.1 types to Python 3 types and includes a subcommand to generate C source code for OER or UPER without dynamic memory allocation.

Tokens
3.4K
Snippets
9
Records
16
Agent score
49%

What's inside asn1tools

  1. Compare asn1tools with other C code generators

    master

    This benchmark compares the C source code generated by asn1tools against asn1scc and asn1c.

    Key differences noted in the benchmark:

    • Executable Size: asn1tools produces significantly smaller executables than asn1c. This is because asn1c includes encode/decode functions for all codecs and supports more ASN.1 constructs using dynamic memory, whereas asn1tools is optimized for UPER.
    • Execution Time: asn1tools demonstrates faster encoding and decoding times compared to asn1scc and asn1c under similar optimization levels (e.g., -O3).
    • Code Complexity: asn1tools has the smallest codebase (832 lines of code), followed by asn1scc (3549 lines), and asn1c (10159 lines).
    • Memory Usage: asn1tools and asn1scc show stable runtime memory usage, while asn1c exhibits more significant heap usage fluctuations during execution.
  2. Mapping ASN.1 programming types to language types

    master

    The programming_types.asn module defines common ASN.1 types that correspond to standard data types in C, Python, and Java. Use these mappings to ensure compatibility when encoding or decoding data intended for specific programming language environments.

    Key definitions:

    • Double is defined as 64-bit floating point.
    • Float is defined as 32-bit floating point.
    | ASN.1      | C             | Python    | Java       |
    |============|===============|===========|============|
    | `Int8`     | `int8_t`      | `int`     | `byte`     |
    | `Int16`    | `int16_t`     | `int`     | `short`    |
    | `Int32`    | `int32_t`     | `int`     | `int`      |
    | `Int64`    | `int64_t`     | `int`     | `long`     |
    | `Uint8`    | `uint8_t`     | `int`     | `byte`     |
    | `Uint16`   | `uint16_t`    | `int`     | `short`    |
    | `Uint32`   | `uint32_t`    | `int`     | `int`      |
    | `Uint64`   | `uint64_t`    | `int`     | `long`     |
    | `Float`    | `float`       | `float`   | `float`    |
    | `Double`   | `double`      | `float`   | `double`   |
    | `Bool`     | `bool`        | `bool`    | `bool`     |
    | `String`   | `char *`      | `str`     | `String`   |
    | `Bytes`    | `uint8_t *`   | `bytes`   | `byte[]`   |
  3. Achieve more compact UPER encoding using OPTIONAL and CHOICE instead of extension markers

    master

    You can achieve more compact Unaligned PER (UPER) encoding by avoiding standard ASN.1 extension markers (...) and instead using OPTIONAL fields and power-of-two CHOICE structures at the end of a message.

    Requirements:

    • This technique only works if the extension is located at the very end of the message.

    Implementation Pattern:

    1. For simple extensions: Replace the extension marker ... with extension NULL OPTIONAL.
    2. For versioned extensions: Wrap new fields in a new SEQUENCE (e.g., v2 SEQUENCE) and ensure that sequence also ends with extension NULL OPTIONAL to allow for even further future extensions.
    3. For scaling beyond four messages: If you need to support more than four message variations in a CHOICE block, you must replace the final NULL member of the choice with a nested CHOICE structure (e.g., more CHOICE {fum Fum, extension1 NULL, extension2 NULL, extension3 NULL}).

    This approach reduces the bit overhead typically associated with standard extension markers.

  4. Map ASN.1 types to Python types

    master

    When working with asn1tools, ASN.1 types are automatically mapped to Python 3 types. Use the following mapping to prepare your Python data for encoding or to interpret decoded results:

    ASN.1 typePython typeExample
    BOOLEANboolTrue
    INTEGERint87
    REALfloat33.12
    NULL--None
    BIT STRINGtuple(bytes, int)(b'\x50', 4)
    OCTET STRINGbytesb'\x44\x1e\xff'
    OBJECT IDENTIFIERstr'1.33.2'
    ENUMERATEDstr or int'one' or 1 (Use numeric_enums=True during compilation for integer values)
    SEQUENCEdict{'a': 52, 'b': 1}
    SEQUENCE OFlist[1, 3]
    SETdict{'foo': 'bar'}
    SET OFlist[3, 0, 7]
    CHOICEtuple(str, object)('a', 5)
    UTF8String / PrintableString / etc.str'hello'
    UTCTime / GeneralizedTimedatetime.datetimedatetime(2018, 6, 11, 11, 4, 59)
    DATEdatetime.datedate(1985, 4, 12)
    TIME-OF-DAYdatetime.timetime(15, 27, 46)
    DATE-TIMEdatetime.datetimedatetime(1985, 4, 12, 15, 27, 30)
  5. Install asn1tools via pip

    master

    Install the asn1tools package using pip to enable ASN.1 parsing, encoding, and decoding in Python.

    pip install asn1tools
  6. Compare compact UPER encoding vs standard extension markers

    master

    The following execution log demonstrates the difference in encoded output size between the compact OPTIONAL/CHOICE method and the standard ASN.1 extension marker (...) method using asn1tools.

    Compact UPER (using OPTIONAL/CHOICE)

    Encode V1:

    • Input: ('foo', {'b': 55, 'e': 'on', 'a': True, 'c': 3, 'd': False}) $\rightarrow$ Output: 16ec (2 bytes)

    Encode V2:

    • Input: ('foo', {'b': 55, 'e': 'on', 'a': True, 'c': 3, 'd': False}) $\rightarrow$ Output: 16ec (2 bytes)
    • Input: ('foo', {'v2': {'b': True, 'a': -1}, 'b': 55, 'd': False, 'c': 3, 'e': 'on', 'a': True}) $\rightarrow$ Output: 36ec3fc0 (4 bytes)
    • Input: ('bar', 3) $\rightarrow$ Output: 46 (1 byte)

    Standard UPER (using extension markers ...)

    Encode V2 with extension markers:

    • Input: ('foo', {'b': 55, 'e': 'on', 'a': True, 'c': 3, 'd': False}) $\rightarrow$ Output: 2dd8 (2 bytes)
    • Input: ('foo', {'b': 55, 'd': False, 'g': True, 'c': 3, 'e': 'on', 'a': True, 'f': -1}) $\rightarrow$ Output: 6dd80204ff00 (6 bytes)
    • Input: ('bar', 3) $\rightarrow$ Output: 800118 (3 bytes)
    # To run the example provided in the repository:
    $ ./main.py
  7. Compile ASN.1 specifications with asn1tools.compile_files

    master

    Use asn1tools.compile_files to compile one or more ASN.1 specification files into a Specification object. This object is then used for encoding and decoding data according to the compiled specification.

    import asn1tools
    spec = asn1tools.compile_files(['path/to/spec.asn'])
  8. Parse ASN.1 encoded data with parse_files, parse_string, or parse_dict

    master

    To decode ASN.1 encoded data, use the following functions depending on your input format:

    • asn1tools.parse_files(files, type_name, ...): Parses encoded data from files.
    • asn1tools.parse_string(string, type_name, ...): Parses encoded data from a string.
    • asn1tools.parse_dict(dict, type_name, ...): Parses encoded data from a dictionary.

    Note: These functions typically require the name of the ASN.1 type you are parsing.

  9. Encode and decode ASN.1 data in Python

    master

    Use asn1tools.compile_files() to compile an ASN.1 specification file. This returns a module object used to encode and decode messages using a specified codec.

    Supported codecs include:

    • Basic Encoding Rules (BER)
    • Distinguished Encoding Rules (DER)
    • Generic String Encoding Rules (GSER)
    • JSON Encoding Rules (JER)
    • Basic Octet Encoding Rules (OER)
    • Aligned Packed Encoding Rules (PER)
    • Unaligned Packed Encoding Rules (UPER)
    • XML Encoding Rules (XER)

    If no codec is specified during compilation, it defaults to BER.

    import asn1tools
    # Compile the specification
    foo = asn1tools.compile_files('tests/files/foo.asn')
    
    # Encode a message (defaulting to BER)
    encoded = foo.encode('Question', {'id': 1, 'question': 'Is 1+1=3?'})
    
    # Decode the message
    decoded = foo.decode('Question', encoded)
    print(decoded)  # {'id': 1, 'question': 'Is 1+1=3?'}
  10. Compile ASN.1 specifications from strings or dictionaries

    master

    For scenarios where the specification is not in a file, use these functions to create a Specification object:

    • asn1tools.compile_string(string, ...): Compiles an ASN.1 specification provided as a string.
    • asn1tools.compile_dict(dict, ...): Compiles an ASN.1 specification provided as a dictionary.
  11. Supported ASN.1 Encodings

    master

    asn1tools supports several ASN.1 encoding rules as specified by ITU-T and IETF standards, including:

    • BER, CER, DER: Basic Encoding Rules, Canonical Encoding Rules, and Distinguished Encoding Rules (X.690).
    • PER: Packed Encoding Rules (X.691).
    • XER: XML Encoding Rules (X.693).
    • OER: Octet Encoding Rules (X.696).
    • GSER: Generic String Encoding Rules (RFC 3641).
    • JER: JSON Encoding Rules.
  12. Use the asn1tools shell for interactive conversion

    master

    The asn1tools shell subcommand provides an interactive environment to compile specifications and convert encoded data between formats.

    By default, the shell uses BER as the input codec and GSER as the output codec (producing human-readable text).

    > asn1tools shell
    $ help
    $ compile tests/files/foo.asn
    $ convert Question <hex_encoded_data>