GATK (Genome Analysis Toolkit)

repository·master·Indexed 24 days ago

https://github.com/broadinstitute/gatk

An open-source framework for genomic analysis that integrates tools from GATK and Picard. It supports massively parallel execution via Apache Spark and provides a streamlined CLI for bioinformatics workflows. The toolkit includes support for local execution, Spark clusters, and Google Cloud Dataproc, with specific requirements for Java 17, Python 3.10.13, and R 4.3.1.

Tokens
21.3K
Snippets
26
Records
105
Agent score
81%

What's inside GATK

  1. Overview of the GATK gCNV computational kernel

    master

    gcnvkernel is a Python module that serves as the computational backend for GATK germline copy number variant (gCNV) tools and workflows. It implements inference schemes for read-depth profile denoising, germline integer copy number variation discovery, and germline contig ploidy determination.

    While intended to be called privately by GATK CLI tools (where its outputs are treated as intermediate data), advanced users can use it directly via provided Python CLI scripts or in interactive Python environments for exploratory analysis.

  2. Understand Funcotator annotation specifications

    master
    Funcotator generates annotations for genomic variants. This documentation provides the technical specifications for the annotations that Funcotator can create, including details for pre-packaged data sources like Gencode. Users can refer to the Funcotator tool documentation or the official GATK forums for tutorials and deeper technical information.
  3. What is Funcotator and how is it used in M2?

    master

    Funcotator (FUNCtional annOTATOR) is a functional annotation tool that analyzes variants and produces results in VCF (with INFO field annotations) or MAF formats. It uses data sources like GENCODE, dbSNP, gnomAD, and COSMIC to classify variants and provide gene information.

    M2 Integration Details:

    • By default, the M2 WDL runs Funcotator to produce a TCGA MAF from the M2 VCF.
    • Metadata: You should pass specific parameters to populate TCGA MAF metadata fields. These are ignored if Funcotator is not run.
    • Limitations: Certain TCGA MAF fields (like validation alleles) cannot be generated by M2/Funcotator and must be handled by downstream processes.
    • Vocabulary: Funcotator does not strictly enforce the TCGA MAF controlled vocabulary; users are responsible for correct specification. Consequently, generated MAFs may not pass the TCGA Validator.
  4. What is Funcotator and how does it work?

    master

    Funcotator (FUNCtional annOTATOR) is a functional annotation tool within the GATK toolset designed for both somatic and germline use cases. It analyzes variants in a VCF file and labels them with one of twenty-three distinct variant classifications.

    Key capabilities include:

    • Producing gene information (e.g., affected gene, predicted amino acid sequence).
    • Associating variants with data sources like GENCODE, dbSNP, gnomAD, and COSMIC.
    • Outputting results in either Variant Call Format (VCF) or Mutation Annotation Format (MAF).
    • Allowing user-defined annotations via extensible and configurable data sources without modifying Java code.
  5. Structure and installation of GATK Python packages

    master

    Python packages used by GATK must be placed in this directory, with each package contained in its own subdirectory.

    • Standalone Packages: If a package can be installed independently, you may place a setup_<PACKAGE_NAME>.py file in this directory.
    • Standard GATK Installation: During the creation of the common GATK conda environment, setup.py combines all packages and pip-installs them as a single package named gatkpythonpackages.
  6. Identify the correct hg19 reference version

    master

    There are four common versions of the 'hg19' reference that are NOT directly interchangeable. When performing genomic analysis, ensure you are using the specific file and MD5sum required by your pipeline or institution to avoid coordinate or sequence discrepancies.

    Reference Versions

    • GRCh37: The baseline human genome reference built by the Genome Reference Consortium. File: GRCh37.p13.genome.fasta (MD5sum: c140882eb2ea89bc2edfe934d51b66cc).
    • hg19 (UCSC): A reference based on GRCh37 created by UCSC. It contains contig name changes and some known sequence differences. File: ucsc.hg19.fasta (MD5sum: a244d8a32473650b25c6e8e1654387d6).
    • b37 (Broad): A reference based on GRCh37 created by the Broad Institute. It contains contig name changes and sequence differences (often masking low-confidence bases with IUPAC symbols). Note: At the Broad Institute's Genomics Platform, 'hg19' typically refers to this b37 version. File: Homo_sapiens_assembly19.fasta (MD5sum: 886ba1559393f75872c1cf459eb57f2d).
    • humanG1Kv37: Equivalent to b37, but excludes the decoy sequence for human herpesvirus 4 type 1 (NC_007605). File: human_g1k_v37.fasta (MD5sum: 0ce84c872fc0072a885926823dcd0338).
  7. Configure Funcotator transcript selection modes

    master

    The --transcript-selection-mode parameter determines how the primary annotated transcript is selected. The available modes are:

    • CANONICAL (Default): Selects a transcript based on canonical order (User custom list > Curation level > Variant classification rank > Appris annotation > Sequence length > Alphabetical).
    • BEST_EFFECT: Selects a transcript based on functional effect (User custom list > Variant classification rank > Curation level > Appris annotation > Sequence length > Alphabetical).
    • ALL: Similar to CANONICAL, but does not drop any transcripts; it renders all overlapping transcripts.
  8. Compare human reference genome versions using MD5sums

    master

    When working with Funcotator or GATK, it is critical to identify which human reference genome version is being used. Differences between versions like GRCh37, hg19, b37, and HumanG1Kv37 are identified by comparing the MD5sums of their contigs.

    In the reference comparison documentation:

    • Contigs with identical MD5sums across references are listed in the table rows.
    • A blank entry (----) for a contig indicates that the MD5sum does not match, signifying a sequence difference between those specific references.
    • Primary contigs with differing MD5sums are highlighted in red.
    • Alternate contigs with differing MD5sums are highlighted in orange.
  9. Module structure of gcnvkernel

    master

    The gcnvkernel module is organized into several functional submodules:

    • gcnvkernel.inference: Variational inference classes (convergence tracking, deterministic annealing, stochastic optimizers).
    • gcnvkernel.io: I/O routines for genomic intervals, read count data, posteriors, and sample metadata.
    • gcnvkernel.models: PyMC model declarations, pytensor symbolic operations (e.g., HMM forward-backward algorithm), and custom distributions.
    • gcnvkernel.preprocess: Interval list filtering routines.
    • gcnvkernel.postprocess: Post-processing classes (e.g., Viterbi segmentation, segment quality calculation).
    • gcnvkernel.structs: Representations for genomic intervals, interval annotations, and sample metadata.
    • gcnvkernel.tasks: Inference task executors and templates for ADVI inference tasks.
    • gcnvkernel.utils: CLI utilities, recursive least squares (RLS) implementation, and mathematical functions.
  10. Gencode Annotation Specification in Funcotator

    master
    Funcotator automatically generates Gencode annotations for all input variants. These annotations provide detailed genomic, transcript, and protein-level information based on the Gencode reference. The annotations include gene symbols, transcript IDs, variant classifications (e.g., MISSENSE, NONSENSE, SPLICE_SITE), and changes in both cDNA and protein sequences.
  11. Understand Funcotator data source structure

    master

    Funcotator uses data sources organized into a specific directory hierarchy. Each main data source folder must contain sub-directories for each supported reference (e.g., hg19, hg38). Inside each reference-specific directory, a .config file is required to detail how to match the data to a variant.

    Directory Structure Example:

        dataSourcesFolder/
             Data_Source_1/
                 hg19
                     data_source_1.config
                     data_source_1.data.file.one
                     ...
                 hg38
                     data_source_1.config
                     data_source_1.data.file.one
                     ...
             Data_Source_2/
                 hg19
                     ...

    Constraints:

    • Multiple data source folders can be specified as input.
    • No two data sources can have the same name.
  12. Create user-defined data sources

    master

    You can add custom annotations by creating a new sub-directory in your main data sources folder. This directory must contain reference-specific folders (e.g., hg38) containing the data files and a mandatory .config file.

    Supported data formats include:

    • simpleXSV: Separated value table (CSV/TSV) keyed by Gene Name or Transcript ID.
    • locatableXSV: Separated value table keyed by genome location (requires contig, start, and stop columns).
    • gencode: GENCODE data files (GTF format).
    • cosmic: COSMIC data.
    • vcf: Variant Call Format files.
    • Cloud Sources: You can also point src_file to a cloud URL (e.g., gs://...) in your config file to avoid local storage.