MetaWRAP Documentation

repository·master·Indexed 19 days ago

https://github.com/bxlab/metawrap

MetaWRAP is a flexible pipeline for genome-resolved metagenomic data analysis. It provides a suite of standalone modules for read QC, assembly (using MegaHit or metaSPAdes), taxonomy profiling (Kraken/Kraken2), binning (MaxBin2, metaBAT2, CONCOCT), bin refinement, reassembly, abundance quantification, and functional annotation. It also includes Blobology for visualizing community and extracted bins via blobplots.

Tokens
33.3K
Snippets
109
Records
142
Agent score
66%

What's inside MetaWRAP

  1. Overview of MetaWRAP modules

    master

    MetaWRAP is a suite of standalone modules for metagenomic analysis. Each module can be used independently depending on your specific needs.

    Metagenomic data pre-processing modules:

    1. Read_QC: Read trimming and host (e.g., human) read removal.
    2. Assembly: Metagenomic assembly and QC using metaSPAdes or MegaHit.
    3. Kraken/Kraken2: Taxonomy profiling and visualization of reads or contigs.

    Bin processing modules:

    1. Binning: Initial bin extraction using MaxBin2, metaBAT2, and/or CONCOCT.
    2. Bin_refinement: Consolidates multiple binning predictions into a superior bin set.
    3. Reassemble_bins: Reassembles bins to improve completion and N50, and reduce contamination.
    4. Quant_bins: Estimates bin abundance across samples.
    5. Blobology: Visualizes the community and extracted bins with blobplots.
    6. Classify_bins: Provides conservative but accurate taxonomy prediction for bins.
    7. Annotate_bins: Functionally annotates genes within a set of bins.
  2. How MetaWRAP bin refinement and reassembly work

    master

    MetaWRAP provides advanced methods for improving the quality of extracted draft genomes (bins).

    Bin Refinement

    MetaWRAP uses a hybrid approach to consolidate multiple binning predictions (e.g., from metaBAT2, CONCOCT, and MaxBin2). The Bin_refinement module outperforms individual binning approaches and other consolidation tools like Binning_refiner or DAS_Tool. Because this module is standalone, you can use your own preferred binning software predictions as input.

    Bin Reassembly

    The Reassemble_bins module allows you to improve bin quality by extracting the reads belonging to each bin and reassembling them with a more permissive, non-metagenomic assembler. This process:

    • Improves the N50 of the bins.
    • Modestly increases bin completion.
    • Drastically reduces contamination.

    Recommendation: Run Reassemble_bins on the final bin set produced by the Bin_refinement module.

  3. Install MetaWRAP via Bioconda or Docker (Not Recommended)

    master

    The Bioconda and Docker distributions are not recommended for most users because they only receive major releases (e.g., v1.1, v1.2) and do not include the latest patches and bug fixes. For the most up-to-date version, use the -c ursky channel via Conda.

    Bioconda Installation:

    conda install -y -c bioconda metawrap

    Docker Installation:

    docker pull quay.io/biocontainers/metawrap:1.2--0
    docker pull quay.io/biocontainers/metawrap:1.2--0
  4. Set up the KRAKEN2 standard database

    master

    As of metaWRAP v1.3.2, Kraken2 is recommended over Kraken1 as it is more compact and faster to build. It requires approximately 120GB of RAM and 128GB of space.

    1. Build the database:
    kraken2-build --standard --threads 24 --db MY_KRAKEN2_DB
    1. Configure metaWRAP: Locate your config-metawrap file using which config-metawrap and set the KRAKEN2_DB variable to the path of your database directory.
    kraken2-build --standard --threads 24 --db MY_KRAKEN2_DB
    
    # In config-metawrap:
    KRAKEN2_DB=/path/to/my/database/MY_KRAKEN2_DATABASE
  5. Assemble metagenomic reads with Assembly

    master

    The metaWRAP::Assembly module assembles metagenomic reads using one of two assemblers:

    • MegaHit (Default): Highly memory-efficient, faster, and scales well with large datasets.
    • metaSPAdes: Generally produces superior assemblies for most samples.

    Key features:

    • Standardization: For MegaHit, the module sorts and formats the assembly to resemble SPAdes output (contig ID, length, and coverage) and sorts contigs by length.
    • Filtering: Discards short scaffolds (<1000bp).
    • Reporting: Generates an assembly report using QUAST.
  6. Quickly install MetaWRAP dependencies

    master

    To prepare a manual installation, you must first set up the required channels and install the dependencies.

    Note: The order of channel configuration is important.

    For Unix/Linux only:

    # Set up channels
    conda config --add channels defaults
    conda config --add channels conda-forge
    conda config --add channels bioconda
    conda config --add channels ursky
    
    # Install dependencies
    conda install --only-deps -c ursky metawrap-mg

    Universal method (works on other systems):

    conda config --add channels defaults
    conda config --add channels conda-forge
    conda config --add channels bioconda
    conda config --add channels ursky
    
    conda install biopython blas=2.5 blast=2.6.0 bmtagger bowtie2 bwa checkm-genome fastqc kraken=1.1 krona=2.7 matplotlib maxbin2 megahit metabat2 pandas prokka quast r-ggplot2 r-recommended salmon samtools=1.9 seaborn spades trim-galore

    Note: You may need to manually install concoct=1.0 and pplacer when using the universal method.

    conda install --only-deps -c ursky metawrap-mg
  7. Create a bmtagger index for host removal

    master

    To remove host reads in the read_qc module, you must create a bmtagger index.

    For Human Hosts (hg38)

    1. Download and merge the hg38 genome:
    mkdir BMTAGGER_INDEX
    cd BMTAGGER_INDEX
    wget ftp://hgdownload.soe.ucsc.edu/goldenPath/hg38/chromosomes/*fa.gz
    gunzip *fa.gz
    cat *fa > hg38.fa
    rm chr*.fa
    1. Index the genome. Crucial: The output files must be named exactly hg38.bitmask and hg38.srprism for metaWRAP to recognize them.
    bmtool -d hg38.fa -o hg38.bitmask
    srprism mkindex -i hg38.fa -o hg38.srprism -M 100000
    1. Configure metaWRAP: In config-metawrap, set BMTAGGER_DB to the path of your BMTAGGER_INDEX directory.

    For Non-Human Hosts

    Follow the same protocol as above. If your index files are named [prefix].srprism and [prefix].bitmask (e.g., pig.srprism and pig.bitmask), you must specify the prefix when running the read_qc module using the -x flag:

    metawrap read_qc -x pig ...
    # Example for human hg38
    mkdir BMTAGGER_INDEX
    cd BMTAGGER_INDEX
    # ... download and cat hg38.fa ...
    bmtool -d hg38.fa -o hg38.bitmask
    srprism mkindex -i hg38.fa -o hg38.srprism -M 100000
    
    # In config-metawrap:
    BMTAGGER_DB=/path/to/your/index/BMTAGGER_INDEX
  8. Manual installation of MetaWRAP (Recommended)

    master

    Manual installation provides the most control over software versions and is recommended for non-Linux64 systems.

    Steps:

    1. Install the prerequisite programs using Conda (see 'Quickly install dependencies' below).
    2. Clone or download the MetaWRAP repository.
    3. Configure the metaWRAP/bin/config-metawrap file.
    4. Add the metaWRAP/bin/ directory to your $PATH (e.g., by editing ~/.bash_profile) or copy the contents of metaWRAP/bin/ to an executable directory like /usr/bin/ or /miniconda2/bin/.
  9. Perform metagenomic binning with Binning

    master

    The metaWRAP::Binning module wraps three binning tools: MaxBin2, metaBAT2, and CONCOCT.

    Workflow:

    1. Indexing & Alignment: Indexes the assembly with bwa-index and aligns paired-end reads from multiple samples using bwa.
    2. Processing: Sorts and compresses alignments with samtools and calculates library insert size statistics.
    3. Abundance Calculation: Uses metaBAT2's jgi_summarize_bam_contig_depths to create a contig abundance table, which is then formatted for the selected binner(s).
    4. Binning: Runs the chosen software (MaxBin2, metaBAT2, or CONCOCT) and creates formatted bin fasta files.
    5. Quality Check (Optional): Runs CheckM (lineage_wf) to estimate bin completion and contamination and generates a summary report.
  10. Install MetaWRAP via Conda (Basic and Environment-specific)

    master

    MetaWRAP supports Linux64. It is strongly recommended to install MetaWRAP within a dedicated Conda environment to avoid conflicts with existing packages, as it installs over 140 dependencies.

    Recommended Setup (Custom Environment):

    # Create and activate environment
    conda create -y -n metawrap-env python=2.7
    source activate metawrap-env
    
    # Configure channels (Order is important)
    conda config --add channels defaults
    conda config --add channels conda-forge
    conda config --add channels bioconda
    conda config --add channels ursky
    
    # Install MetaWRAP
    conda install -y -c ursky metawrap-mg
    
    # Fix CONCOCT warning messages for MetaWRAP 1.2
    conda install -y blas=2.5=mkl

    Basic Setup (Direct Installation):

    # Install Miniconda2
    wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh
    bash Miniconda2-latest-Linux-x86_64.sh
    
    # Configure channels
    conda config --add channels defaults
    conda config --add channels conda-forge
    conda config --add channels bioconda
    conda config --add channels ursky
    
    # Install MetaWRAP
    conda install -y -c ursky metawrap-mg
    
    # Fix CONCOCT warning messages for MetaWRAP 1.2
    conda install -y blas=2.5=mkl
    conda create -y -n metawrap-env python=2.7
    source activate metawrap-env
    conda config --add channels defaults
    conda config --add channels conda-forge
    conda config --add channels bioconda
    conda config --add channels ursky
    conda install -y -c ursky metawrap-mg
    conda install -y blas=2.5=mkl