cyvcf2

repository·main·Indexed 19 days ago

https://github.com/brentp/cyvcf2

A high-performance Python (3.9+) wrapper around htslib for fast parsing of VCF and BCF files. It provides direct access to genomic data via numpy arrays and includes a command-line tool for filtering by region, INFO fields, and individuals. The library features the VCF class for reading, the Variant class for record access, and the Writer class for creating or modifying VCF files, including support for updating headers and modifying genotypes or FORMAT fields.

Tokens
3.3K
Snippets
15
Records
18
Agent score
65%

What's inside cyvcf2

  1. Modify the INFO field in a VCF

    main

    To add new information to the INFO field of a VCF, you must first update the VCF header using add_info_to_header. The header update requires a dictionary containing the keys 'ID', 'Description', 'Type', and 'Number'. After updating the header, use a Writer initialized with the original VCF as a template to write the modified records to a new file.

    from cyvcf2 import VCF, Writer
    vcf = VCF(VCF_PATH)
    
    # adjust the header to contain the new field
    # the keys 'ID', 'Description', 'Type', and 'Number' are required.
    vcf.add_info_to_header({'ID': 'gene', 'Description': 'overlapping gene',
        'Type':'Character', 'Number': '1'})
    
    # create a new vcf Writer using the input vcf as a template.
    fname = "out.vcf"
    w = Writer(fname, vcf)
    
    for v in vcf:
        # Perform user-defined logic to find new info
        genes = get_gene_intersections(v)
        if genes is not None:
            v.INFO["gene"] = ",".join(genes)
        w.write_record(v)
    
    w.close(); vcf.close()
  2. Modify genotypes and the FORMAT field

    main

    To add new data to the FORMAT field or modify genotypes, follow these steps:

    1. Update Header: Use add_format_to_header with the required keys: 'ID', 'Description', 'Type', and 'Number'.
    2. Initialize Writer: Create a Writer using the input VCF as a template.
    3. Set Format Data: Use v.set_format('FIELD_NAME', data_array) to add data to the FORMAT field. If the field type is Integer and Number=1, the data must be an $n imes 1$ numpy array of integers (where $n$ is the number of samples).
    4. Modify Genotypes: You can overwrite genotypes by assigning values to v.genotypes[index]. For example, to set a sample to 'no calls', use [-1]*v.ploidy + [False].
    5. Refresh Genotypes: After modifying individual genotype indices, you must reassign the field to itself (v.genotypes = v.genotypes) to ensure the Variant object reprocesses the changes and subsequent calls (like v.genotype.array()) reflect the updates.

    Note: Modifying v.genotypes[index] directly leaves the Variant object in an inconsistent state until the reassignment step occurs.

    from cyvcf2 import VCF, Writer
    vcf = VCF(VCF_PATH)
    
    # adjust the header to contain the new field
    vcf.add_format_to_header({
        'ID': 'FILTER_CODE',
        'Description': 'Numeric code for filtering reason',
        'Type': 'Integer',
        'Number': '1'
    })
    
    # create a new vcf Writer using the input vcf as a template.
    fname = "out.vcf"
    w = Writer(fname, vcf)
    
    for v in vcf:
        # Example logic to find samples to filter
        indicies, reasons = filter_samples(v)
        if indicies:
            # add the reasons array to the format dictionary at this locus
            v.set_format('FILTER_CODE', reasons)
            for index in indicies:
                # overwrite the genotypes of each filtered locus to be nocalls
                v.genotypes[index] = [-1]*v.ploidy + [False]
            
            # it is necessary to reassign the genotypes field
            # so that the v Variant object reprocess it
            v.genotypes = v.genotypes
    
        w.write_record(v)
    
    w.close(); vcf.close()
  3. Install cyvcf2

    main

    You can install cyvcf2 using pip or uv.

    Using bundled htslib

    If a binary wheel is available for your platform, this is the simplest method:

    pip install cyvcf2
    # or
    uv pip install cyvcf2

    Using system htslib

    If you have already built and installed htslib (version 1.12 or higher), use the CYVCF2_HTSLIB_MODE=EXTERNAL environment variable and install without binaries:

    CYVCF2_HTSLIB_MODE=EXTERNAL pip install --no-binary cyvcf2 cyvcf2
    # or
    CYVCF2_HTSLIB_MODE=EXTERNAL uv pip install --no-binary cyvcf2 cyvcf2

    Building from source (GitHub)

    To build both htslib and cyvcf2 from source:

    git clone --recursive https://github.com/brentp/cyvcf2
    cd cyvcf2
    CYVCF2_HTSLIB_MODE=BUILTIN python -m pip install .
    pip install cyvcf2
  4. Use the cyvcf2 CLI

    main

    The cyvcf2 package provides a command-line interface. You can invoke it in two ways:

    1. If installed as a package: cyvcf2
    2. Without installation: python -m cyvcf2

    The CLI will exit with the exit code returned by the underlying command execution.

    # If installed
    cyvcf2 [args]
    
    # If not installed
    python -m cyvcf2 [args]
  5. Known limitations when writing VCFs

    main

    When using cyvcf2 to write VCF files, be aware of the following current limitations:

    • UTF-8 Encoding: Does not support writing VCFs encoded with UTF-8 containing non-ASCII characters in string-typed FORMAT fields.
    • String Format Fields: Does not support writing string-type FORMAT fields where Number > 1.
  6. Query all records using the VCF instance

    main

    To retrieve all records in a VCF file, call the VCF instance without arguments. Note: This requires an index file (e.g., .tbi or .idx). If no index is available, this will yield zero records.

    vcf = VCF('some.vcf.gz')
    all_vars = list(vcf())
  7. Perform region-queries in cyvcf2

    main

    You can perform targeted queries on specific genomic regions by calling the VCF instance with a region string (e.g., 'CHROM:START-END').

    vcf = VCF('some.vcf.gz')
    for v in vcf('11:435345-556565'):
        print(str(v))
    vcf = VCF('some.vcf.gz')
    for v in vcf('11:435345-556565'):
        print(str(v))
  8. Parse VCF/BCF files with the VCF class

    main

    The VCF class is the primary entry point for parsing VCF and BCF files. You can iterate over variants in a file using a standard loop.

    Important Note on Numpy Arrays: Attributes like variant.gt_ref_depths return numpy arrays that are backed by underlying C data. Once the variant object goes out of scope, these arrays will contain invalid data. To persist the data, you must create a copy using np.array().

    from cyvcf2 import VCF
    import numpy as np
    
    for variant in VCF('some.vcf.gz'):
        # Basic attributes
        print(variant.CHROM, variant.start, variant.end, variant.REF, variant.ALT)
        
        # Accessing sample fields as numpy arrays
        # Note: Use np.array() to copy if you need the data after the loop iteration
        gt_types = variant.gt_types
        gt_depths = np.array(variant.gt_ref_depths) 
        
        # Accessing INFO fields
        dp = variant.INFO.get('DP') # returns int
        fs = variant.INFO.get('FS') # returns float
    
        # Accessing FORMAT fields per sample
        dp_per_sample = variant.format('DP')
    from cyvcf2 import VCF
    
    for variant in VCF('some.vcf.gz'):
        print(variant.CHROM, variant.start, variant.end)
  9. Use the cyvcf2 CLI

    main

    The cyvcf2 command-line tool allows for fast VCF parsing via the terminal.

    Usage: cyvcf2 [OPTIONS] <vcf_file> or -

    Options:

    • -c, --chrom TEXT: Specify what chromosome to include.
    • -s, --start INTEGER: Specify the start of region.
    • -e, --end INTEGER: Specify the end of the region.
    • --include TEXT: Specify what INFO field to include.
    • --exclude TEXT: Specify what INFO field to exclude.
    • --loglevel [DEBUG|INFO|WARNING|ERROR|CRITICAL]: Set the level of log output. [default: INFO]
    • --silent: Skip printing of VCF.
    • --help: Show this message and exit.
    $ cyvcf2 --help
  10. Import core cyvcf2 classes

    main

    The cyvcf2 package provides several primary classes for working with VCF and BCF files. The main entry points are VCF (for reading), Variant (representing individual records), and Writer (for writing files).

    from cyvcf2 import VCF, Variant, Writer
  11. Use the cyvcf2.VCF class to read VCF files

    main

    The cyvcf2.VCF class is the primary interface for reading VCF (Variant Call Format) files. It provides access to variant records and allows for efficient parsing of genomic data.

    from cyvcf2 import VCF
    
    for variant in VCF('example.vcf.gz'):
        print(variant)