SAMtools Documentation

repository·develop·Indexed 24 days ago

https://github.com/samtools/samtools

A suite of tools for manipulating and analyzing high-throughput sequencing data in SAM, BAM, and CRAM formats. SAMtools works in conjunction with htslib and bcftools. The toolkit includes specialized commands such as addreplacerg for managing read group tags, ampliconclip for clipping read alignments, and ampliconstats for producing amplicon sequencing statistics.

Tokens
31.3K
Snippets
53
Records
184
Agent score
81%

What's inside SAMtools

  1. Overview of the SAMtools ecosystem

    develop

    SAMtools is part of a coordinated suite of tools for handling high-throughput sequencing data. It is one of three primary projects:

    • htslib: A C-library for handling high-throughput sequencing data.
    • samtools: Tools for handling SAM, BAM, and CRAM formats (including mpileup).
    • bcftools: Tools for handling VCF and BCF formats (including variant calling).
  2. Build SAMtools from a Git repository

    develop

    Because release tarballs contain generated files not present in the Git repository, you must perform extra steps to build from source.

    By default, the build process looks for an HTSlib source tree in ../htslib. To use a different HTSlib source tree or a previously installed version, use the --with-htslib=DIR flag during the configuration step.

    autoheader            # Build config.h.in
    autoconf -Wno-syntax  # Generate the configure script
    ./configure           # Configure optional functionality (use --with-htslib=DIR to specify HTSlib path)
    make
    make install
  3. Overview of Samtools utilities

    develop

    Samtools is a suite of utilities designed to manipulate sequence alignments in SAM (Sequence Alignment/Map), BAM, and CRAM formats. It supports format conversion, sorting, merging, indexing, and rapid retrieval of reads from specific genomic regions.

    Key characteristics:

    • Stream-oriented: Samtools works on streams. An input file of - represents stdin, and an output file of - represents stdout. This allows for efficient use of Unix pipes.
    • Remote Access: It can open files directly from remote FTP or HTTP(S) servers. It will attempt to download indices automatically if they are not present in the local directory.
    • Index Handling: If an index is not found using the standard filename suffix (e.g., file.bam.bai), it tries without the suffix (e.g., file.bai). You can explicitly link a data file and an index in a different location using the ##idx## notation: /path/to/data.bam##idx##/path/to/index.bai.
  4. Print sample names and reference paths from alignment files

    develop

    The samtools samples command extracts sample names from read-group (@RG) headers and identifies the associated reference genome paths from SAM, BAM, or CRAM files. The output is a tab-delimited list suitable for workflow managers.

    Output Format:

    1. Sample Name: The first column. If no sample is found in the read-group header, a dot (.) is used.
    2. Alignment File Path: The second column.
    3. Reference Path (Optional): The third column. If no reference is found, a dot (.) is used. This column is populated if you provide references via -f or -F.
    4. Index Status (Optional): The fourth column. A single character (Y/N) indicating if the file is indexed. This column is populated if you use the -i flag.
  5. Handle paired-end and singleton reads with samtools fastq

    develop

    When converting paired-end data, samtools fastq categorizes reads based on the READ1 and READ2 flag bits:

    1. Category 1: Only READ1 is set.
    2. Category 2: Only READ2 is set.
    3. Category 0: Both are set, or neither is set (catch-all).

    Managing Output Files:

    • Separate Files: Use -1 FILE for READ1, -2 FILE for READ2, or -o FILE (equivalent to -1 and -2) to write specific categories to files.
    • Singletons: Use -s FILE to write singleton reads (reads that do not have a corresponding pair in the other category) to a specific file. If -s is used, only paired sequences will be output to the standard output/other files for categories 1 and 2.
    • Interleaved/Mixed Output: To output all reads (paired and singletons) into a single stream, redirect the output to a file (e.g., samtools fastq ... > all_reads.fq).
    samtools fastq -0 /dev/null -s single.fq -N in_name.bam > paired.fq
  6. Use samtools rmdup to remove PCR duplicates

    develop

    The samtools rmdup command removes potential PCR duplicates. If multiple read pairs have identical external coordinates, it retains only the pair with the highest mapping quality.

    Important Constraints:

    • Paired-end mode: Only works with FR orientation and requires ISIZE to be correctly set. It does not work for unpaired reads (e.g., orphan reads or ends mapped to different chromosomes).
    • Single-end mode: Must be explicitly enabled using the -s flag.

    If you need to handle unpaired reads (orphan reads or ends on different chromosomes), use Picard's MarkDuplicates instead, as samtools rmdup is limited in this regard.

  7. Configure CRAM compression profiles

    develop

    When using samtools view to output CRAM, you can use the --output-fmt-option to set a compression profile. This simplifies setting multiple encoding options at once.

    Profiles available:

    • fast: Optimized for speed (low compression).
    • normal: The default profile.
    • small: Optimized for smaller file size.
    • archive: Optimized for maximum compression (uses more intensive methods like use_arith).

    Example:

    samtools view -O cram,small -o bar.cram bar.bam
    samtools view -O cram,small -o bar.cram bar.bam
  8. Use Fast Mode in samtools collate

    develop

    Fast mode (-f) is an optimized version of collate that uses an in-memory buffer to write out most pairs as soon as they are found, avoiding excessive temporary file usage.

    Key Characteristics of Fast Mode:

    • Filtering: It outputs only primary alignments that have either the READ1 or READ2 flags set (but not both). Any other alignment records are filtered out. This requires that there are no more than two reads for any given QNAME after filtering.
    • Ordering: Unlike standard mode, fast mode does not randomize the ordering of read pairs. Position-dependent biases may remain in the output.
    • Warning: Do not use fast mode if your downstream tools expect randomly ordered paired reads (e.g., aligners that estimate library insert sizes on batches of reads).

    Tuning Fast Mode:

    • Use -r INT to specify the number of reads to store in memory. Increasing this value uses more memory but allows more pairs to be written out early.
  9. Performance considerations for samtools idxstats

    develop
    While samtools idxstats can run on SAM files, CRAM files, or unindexed BAM files, it will be significantly slower in those cases. When the input is unindexed, the command must read through the entire file to produce the summary statistics instead of using the BAM indices.
  10. Retrieve paired reads with -P, --fetch-pairs

    develop

    The -P, --fetch-pairs option retrieves pairs even when the mate is outside the requested region.

    Requirements and Behavior:

    • Requires an indexed regular file.
    • Automatically enables the multi-region iterator (-M).
    • A region must be specified (via command line or -L).
    • It performs two passes: first scanning the requested region to find mates via RNEXT/PNEXT fields, then a second pass to collect the actual reads.
    • Warning: This option is incompatible with -c, -U, or -p.
    • Requirement: RNEXT and PNEXT fields must be accurate (use samtools fixmate if necessary).
  11. Split statistics by tag using --split

    develop

    The --split TAG option allows you to generate categorized statistics based on a specific SAM tag (e.g., RG for Read Group).

    When using this option, samtools stats will create additional files named <prefix>_<value>.bamstat for every unique value encountered in that tag. You can use --split-prefix STR to define a custom prefix for these files. If no prefix is provided, the input filename is used.

  12. Understand the output sections of samtools ampliconstats

    develop

    The output contains file-specific sections (prefixed with F) and combined sections (prefixed with C). File-specific sections are interleaved and ordered by file, then by statistic type. To extract a specific statistic across all files, use grep.

    Key Output Sections:

    • SS / CSS: Summary statistics (e.g., total amplicons, total files, tool version).
    • AMPLICON: Reformatting of the input BED file showing primer coordinates.
    • FSS / CSS: Summary stats (raw sequences, filtered sequences, failed primer matches, matching sequences).
    • FREADS / CREADS: Count of reads assigned to each amplicon (controlled by -m).
    • FRPERC / CRPERC: Percentage distribution of reads between amplicons.
    • FDEPTH / CDEPTH / FVDEPTH / CVDEPTH: Average read depth per amplicon. VDEPTH variants only include "valid" templates (full-length coverage).
    • FPCOV / CPCOV: Percent coverage per amplicon (controlled by -d).
    • FTCOORD / CTCOORD: Distribution of observed template coordinates.
    • FAMP / CAMP: Counts of templates that are correct length, double length, or treble length.
    • FDP_ALL / CDP_ALL / FDP_VALID / CDP_VALID: Depth per reference base. VALID indicates templates with full-length coverage and matching primers. Uses run-length encoding (controlled by -D).