DOCSEARCHbyAlgolia
Submit Docs

GATK-SV (Structural Variation Discovery Pipeline)

repository·main·Indexed 41 days ago

https://github.com/broadinstitute/gatk-sv

A specialized pipeline for discovering structural variations in Illumina short-read whole-genome sequencing (WGS) data. Version 1.0.0-beta includes WDL workflows, the SV-Shell bash reimplementation for single-sample mode, and the SVTK package for parsing and analyzing SV-related data files.

Tokens
86.2K
Snippets
182
Records
349
Agent score
52%

What's inside GATK-SV

  1. What is GATK-SV?

    main

    GATK-SV is a cloud-based ensemble pipeline designed to discover and annotate all classes of structural variants (SV) from short-read whole genome sequencing (WGS) data.

    It detects the following variant types:

    • Deletions
    • Duplications
    • Multi-allelic copy number variants
    • Balanced inversions
    • Insertions
    • Translocations
    • Complex SVs (including 11 classes of complex SVs composed of multiple breakpoints)

    The pipeline maximizes sensitivity by harmonizing outputs from five specific tools:

    1. Manta
    2. Wham
    3. Scramble
    4. cn.MOPS
    5. GATK-gCNV

    To minimize false positives, the pipeline adjudicates and re-genotypes raw SVs by evaluating sequencing evidence including anomalous paired-end (PE) reads, split reads (SR), read-depth (RD), and B-allele frequencies (BAF).

  3. What is SV-Shell?

    main

    SV-Shell is a platform-agnostic implementation of a single-sample structural variant (SV) genotyping pipeline. It is designed to run end-to-end within a single Docker image, making it compatible with any platform supporting Docker (Azure, AWS, GCP, or on-premises).

    Key capabilities include:

    • Collecting per-sample evidence (read depth, PE/SR, and raw SV calls).
    • Merging evidence with a pre-built reference panel.
    • Clustering, genotyping, and resolving complex multi-breakpoint SVs.
    • Scoring and filtering genotypes using an ML-based recalibrator.
    • Annotating results and producing a single genotyped, filtered, and annotated per-sample VCF with QC metrics.

    Unlike other implementations, SV-Shell is implemented in bash and does not require a WDL execution engine, Terra-platform, or GCP dependencies.

  4. Overview of the GATK-SV preprocessing workflow

    main

    The preprocessing workflow standardizes Structural Variant (SV) calls generated by various algorithms into consistent formats. This allows for downstream comparison and integration of results from different callers.

    Specifically, the workflow converts outputs into:

    • A consistent VCF format for Paired-End (PE) and Split-Read (SR) algorithms.
    • A consistent BED format for depth-based algorithms.

  5. Overview of GATK-SV SV-Shell

    main

    SV-Shell is a pure bash reimplementation of the GATK-SV WDL workflows. It is designed to run the single-sample ("case") mode of GATK-SV outside of the Cromwell workflow engine.

    Each *.sh script in this directory mirrors a specific WDL workflow or task. The top-level driver, single_sample_pipeline.sh, chains these modules together in the same order as the GATKSVPipelineSingleSample.wdl workflow. The pipeline calls a case sample against a pre-built reference panel (including bincov matrices, PE/SR/SD files, gCNV models, standardized VCF tars, etc.) to produce a genotyped, filtered, and annotated per-sample SV VCF.

  6. CleanVcf module overview

    main

    The CleanVcf module performs VCF clean-up operations to prepare complex variants for downstream refinement. Key operations include:

    • Adjusting genotypes on allosomal contigs.
    • Collapsing overlapping CNVs into multi-allelic CNVs.
    • Revising genotypes in overlapping CNVs.
    • Removing redundant CNVs.
    • Stitching large CNVs.
    • VCF formatting clean-up.

    Recommended Workflow Order: GenotypeComplexVariants $\rightarrow$ CleanVcf $\rightarrow$ RefineComplexVariants.

  7. Compare GATK-SV calling modes

    main

    GATK-SV provides three distinct modes for Structural Variant (SV) calling depending on your data availability, scale, and platform requirements:

    ModeDesigned forUse whenScalabilityAvailability
    Cohort ModeBiobank-level joint-calling SVsAll data is available at startHundreds of thousands of WGSExclusively Terra (GCP)
    Single-sample (Terra)Clinical use-casesData arrives in small batchesOne WGS at a timeExclusively Terra (GCP)
    Single-sample (SV-Shell)Clinical use-casesData arrives in small batchesOne WGS at a timePlatform-agnostic (Terra/GCP/AWS/Azure/HPC)

    Choose your mode based on whether you need maximum sensitivity for large cohorts (Joint Calling) or a simpler, rolling-batch approach for smaller studies (Single-Sample).

  8. Understand the GATK-SV Docker automated deployment workflow

    main

    GATK-SV uses a CI/CD pipeline via GitHub Actions to build, test, and publish Docker images. This workflow is designed for users who need to configure their own CI/CD pipeline on a fork of the repository to host images on their own registries.

    The automation is driven by .github/workflows/sv_pipeline_docker.yml and the scripts/docker/build_docker.py script. The workflow consists of three main stages:

    1. Determine build arguments: Identifies which images are affected by changes (incremental build) and generates image tags.
    2. Regression testing: Triggered on pull request commits. It builds the images to ensure they are valid; if the build fails, the job fails and images are discarded.
    3. Publishing: Triggered on merges to main. It builds the images and pushes them to Google Container Registry (GCR) and Azure Container Registry (ACR), then updates the image registry metadata files.

  9. Use EvidenceQC for ploidy estimation and dosage scoring

    main

    EvidenceQC is a module that performs ploidy estimation, dosage scoring, and optionally VCF QC. It is a critical intermediate step in the GATK-SV workflow, typically positioned after GatherSampleEvidence and before Batching and sample filtering.

    Recommended Workflow Order:

    1. GatherSampleEvidence
    2. EvidenceQC
    3. Batching, sample QC, and sex assignment

    Key Capabilities:

    • Ploidy Estimation: Estimates chromosome ploidy and sex assignments. It is recommended to use these estimates to update your PED file (using sex = 0 for sex aneuploidies).
    • Dosage Scoring: Calculates Whole-Genome Dosage (WGD) scores. WGD distributions should be centered around 0 (PCR- samples may be slightly lower, PCR+ samples slightly higher).
    • VCF QC: (Optional) Performs quality control on raw SV call VCFs.
    • Outlier Detection: Identifies samples with unusually low or high SV call counts per contig.
    stateDiagram
  direction LR
  
  classDef inModules stroke-width:0px,fill:#caf0f8,color:#00509d
  classDef thisModule font-weight:bold,stroke-width:0px,fill:#ff9900,color:white
  classDef outModules stroke-width:0px,fill:#caf0f8,color:#00509d

  gse: GatherSampleEvidence
  eqc: EvidenceQC
  batching: Batching, sample QC, and sex assignment
  
  gse --> eqc
  eqc --> batching
  
  class eqc thisModule
  class gse inModules
  class batching outModules

  10. Understand MainVcfQC output structure

    main

    The MainVcfQC module produces a tarball (sv_vcf_qc_output) containing the following directory structure:

    • /plots/main_plots: Primary QC plots for overall call set assessment.
    • /plots/supplementary_plots: A more comprehensive set of detailed QC plots, some of which serve as panels for the main plots.
    • /data: Data files used in the QC process, including family structures, sample lists, variant statistics, and overlap lists against external call sets.

  11. Genotype a batch of samples with GenotypeBatch

    main

    The GenotypeBatch module genotypes a batch of samples across all variants in a cohort. While the preceding MergeBatchSites step is a cohort-level operation, genotyping is performed on one batch of samples at a time.

    How it works

    Genotyping is performed in two main stages:

    1. Training: The module trains variant metric cutoffs on sites that show clear evidence signatures.
    2. Assignment: Genotypes and genotype qualities are assigned using parametric models tuned with these trained cutoffs. This process is performed separately for PE/SR (Paired-End/Split-Read) calls and depth-based calls.

    Recommended Workflow Order

    To use GenotypeBatch correctly, it should be placed in the following pipeline sequence:

    1. MergeBatchSites (Cohort-level merging)
    2. GenotypeBatch (Batch-level genotyping)
    3. RegenotypeCNVs (Downstream CNV processing)
    stateDiagram
  direction LR
  
  classDef inModules stroke-width:0px,fill:#caf0f8,color:#00509d
  classDef thisModule font-weight:bold,stroke-width:0px,fill:#ff9900,color:white
  classDef outModules stroke-width:0px,fill:#caf0f8,color:#00509d

  mbs: MergeBatchSites
  gb: GenotypeBatch
  rgc: RegenotypeCNVs
  mbs --> gb
  gb --> rgc
  
  class gb thisModule
  class mbs inModules
  class rgc outModules

  12. Guidelines for selecting reference panel samples

    main

    When generating a new reference panel for single-sample mode, samples should be chosen to match the criteria used in joint calling batching. This includes maintaining even sex balance and similar depth, WGD score, insert size, and aligner relative to each other and the samples being called.

    Note that the reference panel can only consist of a single batch; multi-batch reference panels are not supported.

    Recommended guidelines for sample selection:

    • Maximize genetic diversity.
    • Use no related samples.
    • Ensure no aneuploidies (germline or mosaic) on allosomes or autosomes.
    • Avoid samples with low sequencing coverage.
    • Avoid samples in the tails of the WGD score distribution.