asammdf Documentation

repository·master·Indexed 21 days ago

https://github.com/danielhrisca/asammdf

A high-performance Python library for parsing, editing, and converting ASAM MDF (Measurement Data Format) files. It supports MDF versions 2, 3, and 4, providing a programmatic API and a graphical user interface for automotive data analysis. Key features include signal filtering, time-interval cutting, version conversion, and support for CAN/LIN bus logging via canmatrix.

Tokens
22.4K
Snippets
45
Records
120
Agent score
71%

What's inside asammdf

  1. Overview of asammdf capabilities

    master

    asammdf is a high-performance parser and editor designed for ASAM (Association for Standardization of Automation and Measuring Systems) MDF (Measurement Data Format) files. It allows developers to programmatically read, edit, and manipulate measurement data.

    Supported MDF Versions

    • MDF 2: .dat files
    • MDF 3: .mdf files
    • MDF 4: .mf4 files
  2. Overview of asammdf features

    master

    The asammdf library is designed for high-performance manipulation of MDF (Measurement Data Format) files.

    Core Capabilities:

    • File Manipulation: Create new MDF files, append channels, merge multiple files, and convert between MDF versions (including v4.10 with zipped data blocks).
    • Data Extraction: Read unsorted MDF v3 and v4 files, extract CAN and LIN signals from bus logging (including anonymous bus logging using .dbc or .arxml databases), and filter subsets of channels.
    • Time Domain Operations: Use the Signal class for unified operations across v3 and v4 files. This is preferred over pandas DataFrames when dealing with channels from different sources at different sampling rates.
    • Export Formats: Export data to pandas, HDF5, Matlab (v7.3), CSV, and parquet.
    • Optimization: Supports space optimizations (no duplicated blocks) and splitting large data blocks for MDF v4.
    • Visualization: Includes a graphical interface to visualize channels and perform file operations.
  3. Understand MDF version 3 block implementations

    master
    The asammdf.blocks.v2_v3_blocks module contains classes that implement the various data blocks used in the MDF version 3 specification. These classes represent the structural components of an MDF file, such as channel definitions, data groups, headers, and identification blocks. While these are low-level components used for version conversion and file parsing, they define how the library handles the internal organization of MDFv3 files.
  4. Access MDF version-specific blocks via the MDF proxy

    master

    The asammdf.mdf.MDF class delegates all attribute access to an underlying _mdf attribute, which is an instance of MDF2, MDF3, or MDF4 depending on the file version.

    • For MDF version 2 & 3, the underlying object is an MDF3 instance (MDF2 and MDF3 share the same implementation).
    • For MDF version 4, the underlying object is an MDF4 instance.

    Users should refer to the MDF3 or MDF4 documentation for specific methods available to those versions.

  5. Rules for writing Python functions in the Functions manager

    master

    When defining functions for virtual channels, you must follow these specific Python syntax rules:

    • Default Numeric Values: All function arguments must have default numeric values.
      • Incorrect: def Function1(a, b, t=0)
      • Correct: def Function1(a=1.5, b=0, t=0)
    • Reserved Time Argument: The last argument must be t=0. This argument is reserved for the time stamp and cannot be used for other purposes.
    • Return Numeric Values: Every possible code branch must explicitly return a numeric value. Returning None or strings will cause errors.
    • Available Modules: The following modules are pre-imported and available within your function:
      • math as math
      • pandas as pd
      • numpy as np
    • Cross-referencing: One user-defined function can call another user-defined function.
    def Function1(a=0, t=0):
        if a > 5:
            return 1
        else:
            return 0
    
    # Function2 can call Function1
    def Function2(b=1, c=7, t=0):
        if b != 0:
            return Function1(c)
        else:
            return c / 2
  6. Configure virtual channel computation modes

    master

    When adding a virtual channel to a Plot window (shortcut Ins), you must choose a Computation mode. This determines how the Python function is executed:

    sample by sample

    • Execution: The function is called $N$ times (once for each time stamp in the union of all signal time bases).
    • Arguments: The function receives individual scalar values for each signal argument and the current time stamp t.
    • Use Case: Best for logic involving conditional branches (e.g., if/else) or simple arithmetic.
    • Limitation: Cannot perform operations that require the full signal context, such as calculating a gradient.

    complete signal

    • Execution: The function is called exactly once.
    • Arguments: The function receives the entire signal as a numpy ndarray for each argument, along with the full time base array t.
    • Use Case: Best for high-performance vector operations using numpy (e.g., np.clip, np.diff).
  7. Understand MDF version 4 block implementations

    master
    The asammdf.blocks.v4_blocks module contains classes that implement the various data structures (blocks) defined in the MDF version 4 specification. These classes represent the low-level components of an MDF file, such as headers, channel definitions, data groups, and attachment information. While most users interact with the high-level ASAMMFD object, these block classes are available for low-level manipulation or inspection of the MDF file structure.
  8. Understand the GUI operating modes

    master

    The GUI operates in three distinct modes that change how files are handled:

    • Single files: Files are opened and viewed individually.
    • Batch processing: Allows for processing multiple files at once.
    • Comparison: Enables viewing and comparing channels from all currently opened files in the same plot.
  9. Understand data coupling in asammdf

    master
    Data returned by MDF methods is decoupled from the original file. Modifying a Signal object returned by a get call does not affect the raw data in the file. Subsequent calls to get for the same channel will always return the original data from the file.
  10. Understand asammdf memory loading modes

    master

    When initializing an asammdf MDF object, you can control how much data is loaded into RAM using the memory parameter. This allows you to balance between processing speed and memory consumption:

    • full: Everything is loaded into RAM. This provides the fastest access to channel data but consumes the most memory.
    • low: Metadata is loaded into RAM, but raw channel data is not. This significantly reduces memory usage at the cost of slower data access (as data must be read from disk when requested).
    • minimum: Uses the full loading strategy but aims for the lowest possible RAM footprint during the object creation process.
  11. Use the Single Files mode for visualization and processing

    master

    The Single files page allows you to open multiple files individually for visualization or processing (e.g., exporting to CSV or HDF5).

    Key features include:

    • Parallel File Access: Open multiple files in parallel using tabs. The tab title shows the short file name, and the tooltip shows the full path.
    • Channel Tree Modes: Display the channel tree as a naturally sorted list, grouped by internal file structure, or showing only selected channels.
    • Channel Selection: Use the channel tree to check/uncheck channels. Only checked channels are used when clicking Create window.
    • Advanced Search: Use Ctrl+F to open the advanced search dialog. It supports wildcard and regex patterns. The "Pattern based window" tab allows filtering channels by name pattern and signal value conditions.
    Shortcut: Ctrl+F (Advanced Search)
  12. Use Numeric windows to inspect signal values

    master

    Numeric windows are designed to handle large numbers of channels and are used to inspect instantaneous signal values or search for specific values.

    Key Capabilities:

    • Display Area: Shows raw and scaled values. Double-click a column header to toggle sorting.
    • Formatting: Switch between physical, hex, and binary formats for integers. Adjust float decimal precision.
    • Timestamp Control: Adjust the timestamp using an input box or a slider.
    • Search Mode: Search for specific values using raw or scaled signal samples. You can use a signal name pattern (wildcard) and specify an operator and target value.
    • Range Editing: Double-click a row to open the range editor for that signal.