nf-core/sarek

repository·master·Indexed 17 days ago

https://github.com/nf-core/sarek

A Nextflow-based bioinformatics workflow for variant calling in whole genome or targeted sequencing data. It supports various species, tumor/normal pairs, and relapses, following GATK best practices for preprocessing. Key capabilities include read processing, alignment (BWA-mem, BWA-mem2, dragmap, Sentieon, or GPU-accelerated parabricks), variant calling (GATK, DeepVariant, Strelka, Manta, etc.), and annotation via SnpEff and Ensembl VEP. The pipeline is designed for portability using Docker and Singularity containers.

Tokens
37.3K
Snippets
88
Records
144
Agent score
70%

What's inside nf-core/sarek

  1. Overview of Sarek workflow capabilities

    master

    Sarek is an open-source, container-based bioinformatics workflow written in Nextflow, designed for the analysis of germline or tumor/normal NGS data (including matched relapses). It supports Whole Genome Sequencing (WGS), Whole-Exome Sequencing (WES), and gene-panel samples.

    Key features include:

    • Supported Data Types: Germline and somatic SNVs/SVs, CNVs, purity, and ploidy.
    • Core Tools: HaplotypeCaller, Strelka, Mutect2, Manta, TIDDIT, ASCAT, and Control-FREEC.
    • Annotation: Supports SNPEff and/or VEP for VCF annotation.
    • Quality Control: Integrated MultiQC reporting.
    • Reproducibility: Uses Docker, Singularity, or Conda to ensure version tracking and portable execution.
    • Deployment: Designed for local nodes, High-Throughput Computing (HTC) clusters, or cloud environments like AWS.
  2. Overview of the CAW (Cancer Analysis Workflow) pipeline

    master

    CAW is an open-source Nextflow pipeline designed to resolve somatic variants from Whole Genome Sequencing (WGS) data. It is specifically built to process normal/tumor sample pairs (and can accommodate relapse samples) by following GATK best practices for preprocessing and utilizing multiple variant callers for comprehensive analysis.

    Key Capabilities:

    • Preprocessing: Aligns, realigns, and recalibrates short-read data for both tumor and normal samples in parallel using GATK best practices.
    • Somatic Variant Calling: Uses MuTect1, MuTect2, and Strelka to identify somatic SNVs and small indels.
    • Structural Variant Calling: Uses Manta to identify structural variants.
    • CNV and Heterogeneity Estimation: Uses ASCAT to estimate sample heterogeneity, ploidy, and Copy Number Variations (CNVs).
    • Quality Control: Provides integrated quality control reports via MultiQC.
    • Flexibility: Can be started from raw FASTQ files, from the realignment step, or directly with specific subsets of variant callers.
    • Output: Merges resulting VCF files for downstream processing while retaining individual caller results.
  3. Introduction to nf-core/sarek

    master

    What is nf-core/sarek?

    nf-core/sarek is a Nextflow-based workflow designed for variant detection on whole genome or targeted sequencing data. While initially optimized for Human and Mouse, it is compatible with any species provided a reference genome is available. It supports tumor/normal pairs and can handle additional relapses.

    Key Features

    • Portability: Built with Nextflow and uses Docker/Singularity containers for easy installation and high reproducibility.
    • Modular Design: Uses Nextflow DSL2 with one container per process, leveraging nf-core/modules where possible.
    • Automated Testing: Every release undergoes continuous integration testing on AWS with full-sized datasets to ensure sensible resource allocation and performance benchmarking.
  4. Overview of Sarek WGS analysis workflow

    master

    Sarek is an open-source Nextflow pipeline designed for Whole Genome Sequencing (WGS) analysis to resolve both germline and somatic mutations. It follows GATK best practices for short-read data preprocessing and supports parallel processing for tumor/normal pairs.

    Key capabilities:

    • Sample Modes: Can process normal samples only, tumor/normal pairs, or complex sets including normal, tumor, and multiple relapse samples.
    • Variant Calling:
      • Germline: Uses Strelka and GATK HaplotypeCaller.
      • Somatic: Uses MuTect2 and Strelka.
      • Structural Variants: Uses Manta.
    • Downstream Analysis: Uses ASCAT to estimate sample heterogeneity, ploidy, and Copy Number Variations (CNVs). VCF files are annotated at the end of the pipeline.
    • Quality Control: Provides QC reports via MultiQC.
    • Resilience: Includes checkpoints to allow restarting the workflow from different states.
    • Reproducibility: Supports execution via Docker and Singularity containers.
  5. Overview of Sarek's WGS analysis capabilities

    master
    Sarek is a portable Open Source Nextflow pipeline designed for Whole Genome Sequencing (WGS) analysis to resolve both germline and somatic mutations. It supports processing normal samples or normal/tumor pairs (including matched relapses). The pipeline is modular and can be extended with additional variant callers. It is compatible with WGS, WES (Whole Exome Sequencing), and gene panels.
  6. What is Sarek and what does it analyze?

    master

    Sarek is a portable, Open Source Nextflow pipeline designed for the analysis of Whole Genome Sequencing (WGS) data to resolve both germline and somatic mutations. It is built using GATK best practices for short-read data preprocessing.

    Key capabilities include:

    • Sample Types: Processes normal samples or normal/tumor pairs (including matched relapses).
    • Input Types: Supports WGS, WES (Whole Exome Sequencing), or gene panels.
    • Reference Genomes: Supports GRCh37 and GRCh38, with the ability to add custom genomes.
    • Portability: Uses Docker and Singularity containers for reproducibility and can run on local nodes, HTC clusters, or cloud environments like AWS.
  7. Access nf-core/sarek documentation

    master
    The documentation for the nf-core/sarek pipeline is organized into specific guides for usage and output interpretation. For general information regarding the installation, configuration, and execution of any nf-core pipeline, refer to the central nf-core website.
  8. Understand the nf-core/sarek output directory structure

    master

    The pipeline generates a structured set of output directories within the specified {outdir}. Most plots and summaries are consolidated in the multiqc directory. All paths provided in the documentation are relative to this top-level results directory.

    A typical directory structure includes:

    • csv/: CSV files generated during processing.
    • multiqc/: MultiQC reports and associated data.
    • pipeline_info/: Information regarding the pipeline execution.
    • preprocessing/: Contains subdirectories for specific preprocessing steps, organized by sample:
      • markduplicates/<sample>/
      • recal_table/<sample>/
      • recalibrated/<sample>/
    • reference/: Reference files used during the run.
    • reports/: Tool-specific quality control reports, organized by tool name (e.g., reports/<tool1>/).
    {outdir}
    ├── csv
    ├── multiqc
    ├── pipeline_info
    ├── preprocessing
    │   ├── markduplicates
    │       └── <sample>
    │   ├── recal_table
    │       └── <sample>
    │   └── recalibrated
    │       └── <sample>
    ├── reference
    └── reports
        ├── <tool1>
        └── <tool2>
    work/
    .nextflow.log
  9. Pre-processing: Alignment, Duplicate Marking, and BQSR

    master

    Before calling variants, raw sequencing data must undergo pre-processing to ensure accuracy:

    Alignment

    Reads are mapped to a specific location in a reference genome. This step is critical for identifying mismatches, insertions, and deletions (INDELs).

    Marking Duplicates

    To prevent biased allele support, non-independent measurements (duplicates) must be identified. Duplicates can be:

    • Library duplicates: Caused by PCR during library preparation.
    • Optical duplicates: Caused by the sequencing instrument reading different clusters.

    An algorithm identifies these using orientation and 5' position, choosing one representative pair and marking the others to be ignored in downstream steps.

    Base Quality Score Recalibration (BQSR)

    Sequencing instruments often have systematic errors. BQSR corrects the reported base quality scores by comparing the estimated error rate with the actual (empirical) error rate observed against the reference.

    The Mathematical Approach:

    1. Calculate empirical error rate ($e_{empirical}$) using a Yates correction: $$e_{empirical} = \frac{n_{mismatches} + 1}{n_{bases} + 2}$$
    2. Convert to Phred-scale ($Q_{empirical}$): $$Q_{empirical} = -10 \times \log_{10}(e_{empirical})$$
    3. Calculate the difference ($\Delta$) between the average reported quality and the empirical quality.
    4. The recalibrated score is the reported score minus the sum of all $\Delta$ values for the bins (e.g., lane, machine cycle, sequencing context) the base belongs to.
  10. Filtering and Annotating Variants

    master

    After calling variants, they must be refined and interpreted.

    Filtering Methods

    • Hard Filtering: Uses pre-defined thresholds for specific annotations (e.g., allele-depth, mapping quality). Best for small sample sizes.
    • Soft Filtering (VQSR): Uses Variant Quality Score Recalibration (VQSR) to infer thresholds from the data distribution and overlap with known, validated variants.
    • Deep Learning: Uses pre-trained neural networks trained on large population databases to identify true variants.

    Annotation

    Annotation adds biological and population context to the final VCF file. Common annotated information includes:

    • Gene and transcript overlap.
    • Potential biological consequences of the variant on transcripts.
    • Population frequency (e.g., Minor Allele Frequency from databases like gnomAD).
  11. Understand the nf-core/sarek codebase architecture

    master

    Sarek follows a hierarchical, modular architecture designed for separation of concerns and reusability. The hierarchy flows from atomic processes to orchestrated workflows:

    Modules (atomic processes) → Subworkflows (composed modules) → Workflow (orchestration)

    Key architectural components include:

    • Modules: Atomic processes, either local to the pipeline or imported from the nf-core module ecosystem.
    • Subworkflows: Composed modules that group multiple processes together.
    • Workflow: The top-level orchestration logic.
    • Configuration: Behavior is driven by configuration via ext.* directives, with resource allocations defined in conf/base.config.
    Modules (atomic processes) → Subworkflows (composed modules) → Workflow (orchestration)
  12. Handle Unique Molecular Identifiers (UMIs)

    master

    Sarek supports two main workflows for processing UMIs: UMI-aware deduplication and consensus read generation.

    UMI-aware Deduplication

    GATK MarkDuplicates automatically uses UMI-aware deduplication if UMIs are present in the RX tag of the BAM/CRAM file. This occurs if you specify --umi_in_read_header true or use --umi_loc and --umi_len.

    Note: GATK MarkDuplicates Spark does not support UMIs.

    Consensus Read Generation

    1. fgbio: If --umi_read_structure is specified, Sarek uses fgbio to generate consensus reads. You must provide the correct structure (e.g., 2M11S+T for forward-only or 2M11S+T 2M11S+T for both reads) based on your kit.
    2. Sentieon: You can enable consensus building within the sentieon_dedup step by setting --sentieon_consensus true. This can be used with or without UMIs.

    Configuration Options

    • --umi_in_read_header <true|false>: Use if UMIs were transferred to the read header (e.g., via bclconvert).
    • --umi_loc <string>: Location of UMI in the read (for fastp extraction).
    • --umi_len <integer>: Length of the UMI.
    • --umi_read_structure <string>: The fgbio read structure.
    • --skip_tools <list>: Use to skip duplicate marking and base quality recalibration if performing post-UMI processing.
    --umi_in_read_header true
    --umi_loc <string>
    --umi_len <integer>
    --umi_read_structure <string>
    --sentieon_consensus true