ThermoRawFileParser

repository·master·Indexed 42 days ago

https://github.com/compomics/thermorawfileparser

A cross-platform utility for converting Thermo RAW mass spectrometry files into open formats including mzML, MGF, and Apache Parquet. It features a CLI for batch conversion, a 'query' subcommand for retrieving spectra in JSON PROXI format, and an 'xic' subcommand for generating chromatograms based on JSON filter inputs. Compatible with .NET 8 and available as self-contained releases or via Docker.

Tokens
2.7K
Snippets
6
Records
10
Agent score
4%

What's inside ThermoRawFileParser

  1. Overview of ThermoRawFileParser

    master

    ThermoRawFileParser is a cross-platform tool designed to read Thermo RAW mass spectrometer files and convert them into common open formats. It is compatible with any platform supporting .NET Core.

    Supported Output Formats:

    • MGF
    • mzML and indexed mzML
    • Apache Parquet

    Core Subcommands:

    • query: Returns one or more spectra in JSON PROXI format based on scan number(s).
    • xic: Returns chromatogram data based on JSON filter input.
  2. Enable file logging in ThermoRawFileParser

    master

    By default, the parser only logs to the console. To enable logging to a file, you must modify the log4net.config file to uncomment the file appender section.

    <log4net>
        <root>
            <level value="INFO" />
            <appender-ref ref="console" />
            <!--<appender-ref ref="file" />-->
        </root>
        <!-- ... existing console appender ... -->
        <!-- Uncomment the section below to enable file logging -->
        <!--<appender name="file" type="log4net.Appender.RollingFileAppender">
            <file value="ThermoRawFileParser.log" />
            <appendToFile value="true" />
            <rollingStyle value="Size" />
            <maxSizeRollbacks value="5" />
            <maximumFileSize value="10MB" />
            <staticLogFileName value="true" />
            <layout type="log4net.Layout.PatternLayout">
                <conversionPattern value="%date [%thread] %level %logger - %message%newline" />
            </layout>
        </appender>-->
    </log4net>
  3. Run ThermoRawFileParser via Docker

    master

    You can run ThermoRawFileParser using a Docker container from BioContainers. Before running, check the latest version tag on the biocontainers/thermorawfileparser/tags page.

    To run the container, use the docker run command, mounting your local directory containing raw files to /data_input inside the container. The entrypoint script is ThermoRawFileParser.sh.

    docker run -i -t -v /home/user/raw:/data_input quay.io/biocontainers/thermorawfileparser:<tag> ThermoRawFileParser.sh --help
  4. Basic usage of ThermoRawFileParser

    master

    ThermoRawFileParser can be used to convert Thermo RAW files into various formats like MGF, mzML, or Parquet. You can specify a single input file using -i or an entire directory using -d. If no output directory is specified, files are written to the input directory.

    When running the framework-based version, use dotnet ThermoRawFileParser.dll instead of the executable.

    # Convert a single file with specific output and metadata formats
    ThermoRawFileParser -i=/path/to/raw_file.raw -o=/path/to/output/ -f=0 -g -m=0
    
    # Minimal usage (single file)
    ThermoRawFileParser -i=/path/to/raw_file.raw
    
    # Minimal usage (directory)
    ThermoRawFileParser -d=/path/to/input_directory/
    
    # Framework-based execution
    dotnet ThermoRawFileParser.dll
  5. Requirements for ThermoRawFileParser

    master

    The requirements for running ThermoRawFileParser depend on the version and the release type you have downloaded.

    For Current Versions

    • Framework-based releases: Require the .NET 8 runtime.
    • Self-contained releases: Available for OSX, Linux, and Windows (do not require a separate .NET runtime installation).
    • For Developers: If you intend to build the tool from source, the .NET 8 SDK is required.

    For Versions Prior to 1.5.0

    • Linux and Mac: Require Mono. If you encounter "assembly not found" errors, install mono-complete.
  6. Configure XIC query JSON input

    master

    The xic subcommand requires a JSON array of query objects. Each object can define different types of filters.

    Supported Filter Types:

    1. M/Z and tolerance: Use mz and tolerance (defaults to ppm).
    2. M/Z range: Use mz_start and mz_end.
    3. Sequence: Use sequence and tolerance (defaults to ppm).
    4. Retention Time: Use rt_start and rt_end.
    5. Scan Filter: Use scan_filter (e.g., ms2, ITMS).
    6. Comment: A free text field preserved in the output.

    Example JSON structure:

    [
        {
            "mz": 488.5384,
            "tolerance": 10,
            "tolerance_unit": "ppm"
        },
        {
            "mz": 575.2413,
            "tolerance": 10,
            "rt_start": 630,
            "rt_end": 660,
            "scan_filter": "ms2"
        },
        {
            "mz_start": 749.7860,
            "mz_end": 750.4,
            "rt_start": 630,
            "rt_end": 660
        },
        {
            "sequence": "TRANNEL",
            "tolerance": 10
        },
        {
            "mz": 1014.5099732499732,
            "rt_start": 14.0600881872,
            "rt_end": 14.4167198290667,
            "tolerance": 5,
            "tolerance_unit": "ppm",
            "comment": "Only ion trap scans",
            "scan_filter": "ITMS"
        }
    ]
  7. Use the xic subcommand to generate chromatograms

    master

    The xic subcommand returns one or more chromatograms based on a JSON input file. The JSON file can contain filters for M/Z (with tolerance), M/Z ranges, or peptide sequences (with tolerance). You can also define retention time (RT) ranges and scan filters.

    XIC Subcommand Options:

    • -i, --input=VALUE: Required. The raw file input.
    • -j, --json=VALUE: Required. The JSON input file containing queries.
    • -b, --output=VALUE: The output file.
    • -o, --output_directory=VALUE: The output directory.
    • -6, --base64: Encodes xic vectors as base64 strings.
    • -p, --print_example: Show a JSON input file example.
    • -s, --stdout: Pipe output to standard output.
    • -w, --warningsAreErrors: Return non-zero exit code for warnings.
    • -l, --logging=VALUE: Logging level (0-4).
    # Run XIC extraction using a JSON query file
    ThermoRawFileParser.exe xic -i=/home/user/data_input/raw_file.raw -j=/home/user/xic_input.json
  8. Reference the ThermoRawFileParser CLI options

    master

    The main executable accepts several options for controlling file conversion, output formats, and extraction details. Note that optional parameters must use the -option=value format.

    Core Options:

    • -h, --help: Prints help.
    • -v, --version: Prints version.
    • -i, --input=VALUE: Required. The raw file input.
    • -d, --input_directory=VALUE: Required. The directory containing raw files.
    • -b, --output=VALUE: The output file.
    • -o, --output_directory=VALUE: The output directory.
    • -s, --stdout: Write to standard output (implies silent logging).

    Format Options:

    • -f, --format=VALUE: Spectra output format. 0 for MGF, 1 for mzML, 2 for indexed mzML, 3 for Parquet, 4 for None. Defaults to indexed mzML.
    • -m, --metadata=VALUE: Metadata format. 0 for JSON, 1 for TXT, 2 for None. Defaults to None.
    • -c, --metadata_output_file=VALUE: Specific metadata output file path.
    • -g, --gzip: GZip the output file.
    • -z, --noZlibCompression: Disable zlib compression for m/z and intensities.

    Extraction & Filtering:

    • -p, --noPeakPicking[=VALUE]: Disable native Thermo peak picking. Can accept a comma-separated list of integers or intervals (e.g., 1,2,3 or 1-3 or 1-).
    • -L, --msLevel=VALUE: Select MS levels (e.g., 1,2,3 or 1-3).
    • -a, --allDetectors: Extract additional detector data (UV/PDA etc).
    • -x, --excludeExceptionData: Exclude reference and exception data.
    • -N, --noiseData: Include noise data in mzML output.
    • -C, --chargeData: Include instrument detected charge states in mzML.

    Advanced & Cloud:

    • -u, --s3_url[=VALUE]: Write directly to S3.
    • -k, --s3_accesskeyid[=VALUE]: S3 access key.
    • -t, --s3_secretaccesskey[=VALUE]: S3 secret key.
    • -n, --s3_bucketName[=VALUE]: S3 bucket name.

    Logging & Errors:

    • -l, --logging=VALUE: Level: 0 (silent), 1 (verbose), 2 (default), 3 (warning), 4 (error).
    • -w, --warningsAreErrors: Return non-zero exit code for warnings.