rust-htslib

repository·master·Indexed 18 days ago

https://github.com/rust-bio/rust-htslib

High-level Rust bindings for HTSlib, providing a Rust API for reading and writing BAM and CRAM files. It includes support for indexed random access via IndexedReader, efficient region access with RecordBuffer, BAM header manipulation, and multi-threaded decompression and compression. Optional features include serde serialization, and HTTP, Amazon S3, and Google Cloud Storage access.

Tokens
17.7K
Snippets
61
Records
78
Agent score
58%

What's inside rust-htslib

  1. Install rust-htslib via Cargo

    master

    To use rust-htslib in your project, add it to your Cargo.toml dependencies. By default, it includes support for CRAM files via bzip2-sys and lzma-sys.

    If you do not require CRAM support and want to reduce your dependency count, you can disable default features.

    # Standard installation with CRAM support
    [dependencies]
    rust-htslib = "*"
    
    # Minimal installation without CRAM support
    [dependencies]
    rust-htslib = { version = "*", default-features = false }
  2. System requirements for rust-htslib

    master

    Building rust-htslib requires a C toolchain compatible with the cc crate. The build script will automatically build and link htslib.

    Pre-built bindings for htslib are available for Mac and Linux. If you are on another platform or require custom bindings, you may need to use the bindgen feature.

  3. How to read tabix-indexed text files

    master

    The tbx module provides a Reader for reading tabix-indexed text files (like BED) in a line-based, format-agnostic way.

    To use it, follow this general workflow:

    1. Create a tbx::Reader using from_path or from_url.
    2. Resolve a chromosome/contig name to its numeric ID using tid(name).
    3. Seek to a specific genomic region using fetch(tid, start, end).
    4. Iterate over the records in that region using the records() iterator or the more efficient read(&mut record) method.

    Note: For accessing tabix-indexed VCF files, it is recommended to use the bcf module instead, as tbx only provides raw lines which require manual parsing.

    use rust_htslib::tbx::{self, Read};
    
    // Create a tabix reader for reading a tabix-indexed BED file.
    let path_bed = "file.bed.gz";
    let mut tbx_reader = tbx::Reader::from_path(&path_bed)
        .expect(&format!("Could not open {}", path_bed));
    
    // Resolve chromosome name to numeric ID.
    let tid = match tbx_reader.tid("chr1") {
        Ok(tid) => tid,
        Err(_) => panic("Could not resolve 'chr1' to contig ID"),
    };
    
    // Set region to fetch (0-based start and end).
    tbx_reader
        .fetch(tid, 0, 100_000)
        .expect("Could not seek to chr1:1-100,000");
    
    // Read through all records in region.
    for record in tbx_reader.records() {
        // ... actually do some work
    }
  4. Represent genotypes with GenotypeAllele

    master

    Genotypes are represented using the GenotypeAllele enum, which handles phased and unphased alleles as well as missing data:

    • GenotypeAllele::Unphased(i32)
    • GenotypeAllele::Phased(i32)
    • GenotypeAllele::UnphasedMissing
    • GenotypeAllele::PhasedMissing

    Note: GenotypeAllele::from_encoded(i32) is deprecated. Use the From<i32> implementation instead.

    A Genotype is a wrapper around a Vec<GenotypeAllele>.

  5. Configure rust-htslib features

    master

    You can enable specific features in rust-htslib to extend its functionality:

    • serde: Enables (de)serialization of bam::Record using any serde-supported format.
    • curl: Enables HTTP access to files.
    • s3: Enables beta-level Amazon S3 support.
    • gcs: Enables beta-level Google Cloud Storage support.
    • bindgen: Instructs hts-sys to generate bindings for your specific architecture using bindgen. Note that this can significantly increase build times. Pre-built bindings are provided for Mac and Linux, but bindgen on Windows is untested.
    [dependencies]
    rust-htslib = { version = "*", features = ["serde"] }
  6. Create and share an HTSlib ThreadPool

    master

    The ThreadPool struct provides a way to manage a pool of threads for parallel processing in HTSlib. You can create a single ThreadPool and share it across multiple BAM readers and writers using set_thread_pool() methods (available on those respective objects).

    Because the Rust wrapper uses Rc (Reference Counting) internally, you do not need to manually manage the lifetime of the ThreadPool; it will be automatically cleaned up when the last reader, writer, or handle referencing it is dropped.

    // Example conceptual usage
    let pool = ThreadPool::new(4)?;
    
    // The pool can then be shared with readers/writers
    // reader.set_thread_pool(&pool);
    // writer.set_thread_pool(&pool);
  7. Access BCF/VCF INFO tags

    master

    The Info struct allows you to access metadata tags in a BCF/VCF record.

    Critical Safety Warning: Methods that return BufferBacked data (like integer(), float(), and string()) return views into an internal buffer. You must keep the BufferBacked object in scope as long as you are accessing the underlying data. If the BufferBacked object is dropped while you are still using the data, you will encounter memory unsafety (accessing unallocated memory).

    // Example of accessing an integer INFO tag
    if let Some(val) = record.info("MY_TAG").unwrap().integer()? {
        // 'val' is a BufferBacked object. It must outlive its usage.
        for &num in val.iter() {
            println!("{}", num);
        }
    }
  8. Access per-sample FORMAT data

    master

    The Format struct is used to access sample-specific data (the columns in a VCF/BCF file).

    Critical Safety Warning: Similar to Info tags, methods returning BufferBacked (like integer(), float(), and string()) tie the lifetime of the data to the BufferBacked object. Ensure the returned object is kept in scope while accessing the data to avoid memory unsafety.

    // Example of accessing integer FORMAT data for all samples
    let fmt = record.format("DP").unwrap();
    let dp_values = fmt.integer()?;
    for sample_data in dp_values.iter() {
        // sample_data is a slice of integers for one sample
        println!("DP: {:?}", sample_data);
    }
  9. Iterate over BAM records

    master

    There are several ways to iterate over records in a BAM file depending on your needs:

    1. Records<'a, R>: A standard iterator over the records of a BAM file.
    2. RcRecords<'a, R>: An iterator that yields Rc<record::Record>, useful if you need to share record ownership.
    3. ChunkIterator<'a, R>: An iterator that stops once the reader's virtual offset reaches a specified end position.
    // Standard iteration
    for record_result in bam_reader.records() {
        let record = record_result?;
        // process record
    }
  10. Use RecordBuffer for efficient BCF region access

    master

    The RecordBuffer provides a way to access specific genomic regions in a sorted BCF file while iterating in a single pass. It uses an internal ringbuffer implementation to allow moving the window to the right with linear complexity.

    Important Constraints:

    • The buffer does not support indexed random access. To access a region at the very end of a BCF file, you must read all preceding records.
    • When using fetch, the start coordinate must be to the right of the start coordinate used in any previous fetch operation.
    • Coordinates are 0-based, and the end coordinate is exclusive.
    let reader = bcf::Reader::from_path("path/to/file.bcf")?;
    let mut buffer = RecordBuffer::new(reader);
    
    // Fetch variants in chromosome '1' from position 100 to 10023
    let (added, deleted) = buffer.fetch(b"1", 100, 10023)?;
    
    // Iterate over the fetched records
    for record in buffer.iter() {
        println!("Position: {}", record.pos());
    }
  11. How `Reader` and `build` work together for FASTA indexing

    master

    To perform random access on a FASTA file, you must first ensure an index exists. The build function generates the index file from the FASTA file. Once indexed, the Reader can be used to efficiently fetch specific genomic coordinates without loading the entire file into memory.

    1. Build: Call build(path) to create the index.
    2. Read: Use Reader::from_path(path) to open the file and its index.
    3. Query: Use fetch_seq or fetch_seq_string to extract specific sub-sequences.
    use rust_htslib::faidx::{build, Reader};
    use std::path::PathBuf;
    
    let path = PathBuf::from("genome.fa");
    
    // 1. Ensure index exists
    build(&path).expect("Failed to build fasta index");
    
    // 2. Open the reader
    let reader = Reader::from_path(&path).expect("Failed to open faidx");
    
    // 3. Fetch data
    let seq = reader.fetch_seq_string("chr1", 100, 200).unwrap();
  12. Create a VCF/BCF Record

    master

    New Record instances should not be constructed manually. Instead, use the empty_record() method provided by bcf::Reader or bcf::Writer. This ensures the record is correctly initialized with the associated header.

    use rust_htslib::bcf::{Format, Writer};
    use rust_htslib::bcf::header::Header;
    
    // Create minimal VCF header with a single sample
    let mut header = Header::new();
    header.push_sample("sample".as_bytes());
    
    // Write uncompressed VCF to stdout with above header and get an empty record
    let mut vcf = Writer::from_stdout(&header, true, Format::Vcf).unwrap();
    let mut record = vcf.empty_record();