pbmm2

repository·develop·Indexed 42 days ago

https://github.com/pacificbiosciences/pbmm2

A SMRT C++ wrapper for the minimap2 C API designed specifically for PacBio data. It provides optimized alignment, indexing, and sorting capabilities, serving as the official replacement for BLASR. It supports various input formats including BAM, FASTA, FASTQ, and XML datasets, with specialized presets for SUBREAD, CCS/HIFI, and ISOSEQ workflows.

Tokens
4.2K
Snippets
12
Records
24
Agent score
53%

What's inside pbmm2

  1. Typical pbmm2 workflows

    develop

    Depending on your input data and requirements, use one of the following common patterns:

    • Pre-indexing references: Index a reference once to reuse it for multiple alignments, which is faster.
    • On-the-fly sorting: Align reads and sort the output in a single step using --sort.
    • XML datasets: Align PacBio subreadsets or consensus readsets using .xml files.
    • Streaming output: Omit the output file argument to stream BAM data to stdout for piping into other tools like samtools.
    • CCS FASTQ input: Align CCS data from FASTQ files using the CCS preset.
    # A. Generate index file for reference and reuse it to align reads
    $ pbmm2 index ref.fasta ref.mmi --preset SUBREAD
    $ pbmm2 align ref.mmi movie.subreads.bam ref.movie.bam --preset SUBREAD
    
    # B. Align reads and sort on-the-fly, with 4 alignment and 2 sort threads
    $ pbmm2 align ref.fasta movie.subreads.bam ref.movie.bam --preset SUBREAD --sort -j 4 -J 2
    
    # C. Align reads, sort on-the-fly, and create PBI
    $ pbmm2 align ref.fasta movie.subreadset.xml ref.movie.alignmentset.xml --preset SUBREAD --sort
    
    # D. Omit output file and stream BAM output to stdout
    $ pbmm2 align hg38.mmi movie1.subreadset.xml --preset SUBREAD | samtools sort > hg38.movie1.sorted.bam
    
    # E. Align CCS fastq input and sort output
    $ pbmm2 align ref.fasta movie.Q20.fastq ref.movie.bam --sort --rg '@RG\tID:myid\tSM:mysample'
  2. Get alignment statistics and progress reports

    develop

    You can retrieve detailed execution information by adjusting the --log-level flag:

    Alignment Metrics (--log-level INFO)

    After alignment, pbmm2 outputs metrics including:

    • Mapped Reads
    • Alignments
    • Mapped Bases
    • Mean Sequence Identity
    • Max/Mean Mapped Read Length

    Timing and Memory (--log-level INFO)

    Provides breakdown of: Index Build/Read Time, Alignment Time, Sort Merge Time, BAI/PBI Generation Time, Run Time, CPU Time, and Peak RSS.

    Progress Reports (--log-level DEBUG)

    Provides real-time updates in the format: #Reads, #Aln, #RPM: [count], [count], [reads per minute]

  3. Configure BAM indexing behavior

    develop

    When using --sort to produce sorted output, pbmm2 generates a BAM index file by default. You can control the index type or disable it using the --bam-index flag:

    • --bam-index BAI: Generates a .bai file (default).
    • --bam-index CSI: Generates a .csi file (recommended for larger genomes).
    • --bam-index NONE: Skips index generation completely.
  4. Configure sorting memory and temporary files

    develop

    When using the --sort option, you can manage memory and storage:

    • Sort memory (-m, --sort-memory): Defines memory allocated per sort thread. Accepts suffixes M (Megabytes) and G (Gigabytes).
    • Temporary files: Stored in the current working directory unless the TMPDIR environment variable is set.

    Best Practice: It is recommended to provide more memory to a few sort threads (rather than less memory to many threads) to avoid disk IO pressure. Benchmarks suggest 4 sort threads are typically optimal, with a maximum effective limit of 8.

    # Example using 2G of memory per sort thread
    $ pbmm2 align ref.fasta movie.subreads.bam ref.movie.bam --sort -m 2G
  5. Configure pbmm2 parameter sets and overrides

    develop

    By default, pbmm2 uses recommended parameter sets to simplify configuration. The available presets are:

    • SUBREAD
    • CCS or HIFI (default)
    • ISOSEQ
    • UNROLLED

    Presets vary by version; check --help for details. You can override specific parameters from your chosen set using the following flags:

    Alignment Parameters

    • -k: k-mer size (max 28). Default: -1.
    • -w: Minimizer window size. Default: -1.
    • -u: Disable homopolymer-compressed k-mer (active for SUBREAD & UNROLLED).
    • -A: Matching score. Default: -1.
    • -B: Mismatch penalty. Default: -1.
    • -z: Z-drop score. Default: -1.
    • -Z: Z-drop inversion score. Default: -1.
    • -r: Bandwidth for chaining and DP-based alignment. Default: -1.
    • -g: Stop chain elongation if no minimizers in N bp. Default: -1.

    Gap Penalty Overrides

    For piece-wise linear gap penalties (where a k-long gap costs min{o+k*e,O+k*E}):

    • -o, --gap-open-1: Gap open penalty 1. Default: -1.
    • -O, --gap-open-2: Gap open penalty 2. Default: -1.
    • -e, --gap-extend-1: Gap extension penalty 1. Default: -1.
    • -E, --gap-extend-2: Gap extension penalty 2. Default: -1.
    • -L, --lj-min-ratio: Long join flank ratio. Default: -1.

    ISOSEQ Specific Overrides

    • -G: Max intron length (affects -r). Default: -1.
    • -C: Cost for non-canonical GT-AG splicing. Default: -1.
    • --no-splice-flank: Do not prefer splice flanks GT-AG.
  6. Use the pbmm2 CLI for indexing and alignment

    develop

    The pbmm2 executable provides two primary subcommands for genomic data processing: index and align.

    Subcommands

    • index: Used to generate an index file from a reference genome (e.g., FASTA) for faster subsequent alignments.
    • align: Used to align sequencing reads (BAM, XML, or FASTQ) against a reference genome or a pre-computed index.

    Typical Workflows

    1. Index then Align (Recommended for reuse): Generate an .mmi index file first, then use it for alignment to save time on multiple runs.

    $ pbmm2 index ref.fasta ref.mmi
    $ pbmm2 align ref.mmi movie.subreads.bam ref.movie.bam

    2. Direct Alignment (On-the-fly): Align directly against a FASTA file. You can use the --sort flag to sort the output BAM on-the-fly.

    $ pbmm2 align ref.fasta movie.subreads.bam ref.movie.bam --sort -j 4 -J 2

    Note: -j specifies alignment threads and -J specifies sort threads.

    3. Align CCS FASTQ input: Use the --preset CCS flag when aligning CCS (Circular Consensus Sequencing) FASTQ files.

    $ pbmm2 align ref.fasta movie.Q20.fastq ref.movie.bam --preset CCS --sort --rg '@RG\tID:myid\tSM:mysample'

    4. Stream output to stdout: Omit the output file argument to pipe the BAM output to another tool like samtools.

    $ pbmm2 align hg38.mmi movie1.subreadset.xml | samtools sort > hg38.movie1.sorted.bam
    # Example: Indexing a reference
    $ pbmm2 index ref.fasta ref.mmi
    
    # Example: Aligning reads using the index
    $ pbmm2 align ref.mmi movie.subreads.bam ref.movie.bam
  7. Known issue: Non-deterministic alignment ordering

    develop
    Due to multithreading, the output alignment ordering may differ between multiple runs using the same input parameters. This non-determinism can occur even when using the --sort option for records that share the same target sequence, position, and orientation (the fields used by samtools sort).
  8. Set read groups for FASTA/Q input

    develop

    When using FASTA or FASTQ input, use the --rg option to set the read group information. This is necessary for downstream compatibility.

    Example: pbmm2 align hg38.fasta movie.Q20.fastq hg38.movie.bam --rg '@RG\tID:myid\tSM:mysample'

    $ pbmm2 align hg38.fasta movie.Q20.fastq hg38.movie.bam --rg '@RG\tID:myid\tSM:mysample'
  9. Use File Of File Names (.fofn) for multiple inputs

    develop

    To process multiple files at once, you can use a .fofn (File Of File Names) file. This text file should contain a list of paths to files of the same datatype (BAM, FASTA, or FASTQ).

    Example:

    # Create a fofn file
    echo "file1.fastq.gz" > myfiles.fofn
    echo "file2.fastq.gz" >> myfiles.fofn
    
    # Run alignment using the fofn
    $ pbmm2 align ref.fasta myfiles.fofn output.bam
    echo "m64001_190131_212703.Q20.fastq.gz" > myfiles.fofn
    echo "m64001_190228_200412.Q20.fastq.gz" >> myfiles.fofn
    pbmm2 align hg38.fasta myfiles.fofn hg38.myfiles.bam --rg '@RG\tID:myid\tSM:mysample'
  10. Supported input and output file types

    develop

    The following table describes supported file formats and their compatibility with GenomicConsensus (GC) and recommended --preset values.

    InputOutputGCPreset
    .bam (aligned or unaligned).bamY
    .fasta / .fa / .fasta.gz / .fa.gz.bamN
    .fastq / .fq / .fastq.gz / .fq.gz.bamN
    .Q20.fastq / Q20.fastq.gz.bamNCCS
    bam.fofn.bamN
    fasta.fofn.bamN
    fastq.fofn.bamN
    .subreadset.xml.bam / .alignmentset.xmlY
    .consensusreadset.xml.bam / .consensusalignmentset.xmlYCCS
    .transcriptset.xml.bam / .transcriptalignmentset.xmlYISOSEQ
  11. New features in pbmm2 v26.2.99

    develop

    The latest version (26.2.99) introduced several performance and functional improvements:

    • Speed Improvements: Enhanced speed for single unaligned BAM inputs using pbsamoa.
    • Chunking: Added --chunk and --chunk-mode options.
    • Batching: Renamed --chunk-size to --batch-size.
    • Tag Management: Added --strip-tags.
    • Indexing: Parallelized .bai generation.