HapHiC: Allele-Aware Hi-C Scaffolding Tool

repository·main·Indexed 42 days ago

https://github.com/zengxiaofei/haphic

A fast, reference-independent, allele-aware scaffolding tool that uses Hi-C data to order and orient contigs into chromosomes. It specifically supports phased hifiasm assemblies and utilizes a four-step pipeline consisting of MCL clustering, contig reassignment, ordering/orientation via 3D-DNA and ALLHiC, and final scaffold construction. Features include support for GFA files, Juicebox visualization, and a refsort command for ordering scaffolds according to a reference genome.

Tokens
9.5K
Snippets
22
Records
32
Agent score
5%

What's inside HapHiC

  1. HapHiC Overview and Features

    main

    HapHiC is a fast, reference-independent, allele-aware scaffolding tool designed to scaffold haplotype-phased genome assemblies into chromosome-scale pseudomolecules using Hi-C data.

    Key Advantages:

    • Reference-Independent: Does not require a reference genome.
    • Allele-Aware: Specifically designed for haplotype-phased assemblies, utilizing phasing information (e.g., from hifiasm).
    • Robustness: High tolerance to low contig N50, low Hi-C sequencing depth, chimeric contigs, collapsed contigs, and switch errors.
    • Efficiency: Super-fast and memory-efficient; supports 'quick view mode' to order/orient contigs without knowing the chromosome number.
    • Compatibility: Supports chromap (pairs format) and produces customizable contact maps.
  2. Use HapHiC Quick View mode

    main

    The --quick_view mode is a lightweight alternative to the full clustering pipeline. It uses a quicksort algorithm to sort and orient all contigs without performing clustering.

    Use this mode when:

    • You are unsure of the exact chromosome number.
    • The full pipeline fails or provides unacceptable results.
    • You need a quick overview of the assembly quality/errors.
    • You want to manually adjust chromosomes in Juicebox.

    Behavior:

    • The nchrs parameter is ignored.
    • Most other parameters are disabled.
    • You can still use --correct_nrounds to perform assembly error correction.
    • For hifiasm phased assemblies, you can still use --gfa to partition contigs into different haplotypes.
    # Quick view (nchrs is ignored)
    $ /path/to/HapHiC/haphic pipeline asm.fa HiC.filtered.bam nchrs --quick_view
    
    # Quick view with assembly error correction
    $ /path/to/HapHiC/haphic pipeline asm.fa HiC.filtered.bam nchrs --quick_view --correct_nrounds 2
    
    # Quick view with haplotype partitioning via GFA
    $ /path/to/HapHiC/haphic pipeline allhaps.fa HiC.filtered.bam nchrs --quick_view --gfa "hap1.p_ctg.gfa,hap2.p_ctg.gfa"
  3. How the HapHiC pipeline works

    main

    HapHiC follows a four-step pipeline to scaffold genomes using Hi-C data:

    1. Clustering: Uses the Markov cluster (MCL) algorithm to group contigs. It automatically iterates through different inflation values to find the best fit for the expected number of chromosomes (nchrs).
    2. Reassignment: Rescues unassigned contigs and uses AHC to merge excess clusters to match the target nchrs.
    3. Ordering and Orientation: Performs fast sorting (3D-DNA style) followed by ALLHiC optimization to determine the linear order and direction of contigs within groups.
    4. Building Scaffolds: Generates final FASTA scaffolds and AGP files (both SALSA-style for corrected assemblies and YaHS-style for raw assemblies) and provides a script for Juicebox visualization.
  4. Step 1: Cluster contigs using the MCL algorithm

    main

    The clustering step performs preprocessing (correcting misjoins, filtering short contigs, and removing allelic Hi-C links) and then uses the Markov cluster (MCL) algorithm to group contigs. Unlike AHC, MCL uses an "inflation" parameter to control clustering granularity. HapHiC automatically tests a range of inflation values (defined by --min_inflation, --max_inflation, and --inflation_step) and recommends the "best" one based on the expected number of chromosomes nchrs and group length distribution.

    Main outputs:

    • corrected_asm.fa: Corrected assembly (if assembly correction is enabled).
    • corrected_ctgs.txt: List of corrected contig IDs.
    • full_links.pkl: Binary file of Hi-C links between contig pairs.
    • HT_links.pkl: Binary file of Hi-C links between half-contig pairs.
    • paired_links.clm: Text file of positional information for paired links.
    • inflation_*: Directories containing group information (group*.txt) and MCL results (mcl_inflation_*.clusters.txt).

    Check HapHiC_cluster.log to find the recommended inflation value (e.g., You could try inflation from 1.20...).

    /path/to/HapHiC/haphic cluster asm.fa HiC.filtered.bam nchrs
  5. Align Hi-C data to the assembly

    main

    Before running the HapHiC pipeline, you must prepare a BAM file by aligning Hi-C data to your assembly.

    Recommended Workflow:

    1. Index the assembly using bwa.
    2. Align reads using bwa mem with specific flags (-5SP) to handle Hi-C data.
    3. Use samblaster to mark PCR duplicates.
    4. Filter the BAM file using samtools to remove secondary/supplementary alignments and low-quality mappings.
    5. Use the HapHiC utility filter_bam to filter by mapping quality (MAPQ) and edit distance (NM).

    Important Notes:

    • asm.fa can be haplotype-collapsed contigs, haplotype-phased unitigs, or haplotype-resolved contigs.
    • DO NOT sort the BAM file by coordinate. If it is already coordinate-sorted, resort it by read name using samtools sort -n.
    • The Juicer pipeline is NOT recommended for haplotype-phased assemblies.
    # (1) Align Hi-C data to the assembly, remove PCR duplicates and filter out secondary and supplementary alignments
    $ bwa index asm.fa
    $ bwa mem -5SP -t 28 asm.fa /path/to/read1_fq.gz /path/to/read2_fq.gz | samblaster | samtools view - -@ 14 -S -h -b -F 3340 -o HiC.bam
    
    # (2) Filter the alignments with MAPQ 1 (mapping quality ≥ 1) and NM 3 (edit distance < 3)
    $ /path/to/HapHiC/utils/filter_bam HiC.bam 1 --nm 3 --threads 14 | samtools view - -b -@ 14 -o HiC.filtered.bam
  6. Install HapHiC via Conda

    main

    To install HapHiC, clone the repository from GitHub and use conda to create an environment with the required dependencies. It is strongly recommended to use the provided Conda environment files rather than the Bioconda version, as the Bioconda version is not officially maintained and may cause workflow failures.

    1. Clone the repository: git clone https://github.com/zengxiaofei/HapHiC.git
    2. Create the environment (using the Python 3.10 configuration): conda env create -f HapHiC/conda_env/environment_py310.yml
    3. Activate the environment: conda activate haphic
    $ git clone https://github.com/zengxiaofei/HapHiC.git
    $ conda env create -f HapHiC/conda_env/environment_py310.yml
    $ conda activate haphic
  7. Step 3: Sort and orient contigs

    main

    This step uses an optimized 3D-DNA iterative scaffolding algorithm (fast sorting) followed by an improved ALLHiC algorithm to optimize contig order and orientation. It uses the --processes flag to set the number of parallel processes.

    Note: If assembly error correction was performed, use corrected_asm.fa instead of asm.fa.

    Key Outputs:

    • group*.tour.sav: Fast sorting results for each group.
    • group*.tour: Optimized contig order and orientation results.
    • final_tours/group*.tour: Final optimized results.
    /path/to/HapHiC/haphic sort asm.fa HT_links.pkl split_clms final_groups/group*.txt --processes 8
  8. Step 4: Build scaffolds

    main

    The final step constructs scaffolds based on the chromosome assignment, order, and orientation from the group*.tour files.

    Scenario A: Without assembly error correction

    /path/to/HapHiC/haphic build asm.fa asm.fa HiC.filtered.bam final_tours/group*.tour

    Scenario B: With assembly error correction You must use corrected_asm.fa as the first FASTA input and provide the corrected_ctgs.txt list via --corrected_ctgs. Failing to do this may result in incorrect scaffolds.raw.agp files.

    /path/to/HapHiC/haphic build corrected_asm.fa asm.fa HiC.filtered.bam final_tours/group*.tour --corrected_ctgs corrected_ctgs.txt

    Key Outputs:

    • scaffolds.agp: SALSA-style AGP file (uses corrected_asm.fa and assigns new IDs to split contigs).
    • scaffolds.raw.agp: YaHS-style AGP file (uses asm.fa; split contigs retain original IDs but show start/end coordinates in columns 7 and 8).
    • scaffolds.fa: Final scaffold sequences in FASTA format.
    • juicebox.sh: Shell script for Juicebox visualization and manual adjustment.
    # Example with error correction
    /path/to/HapHiC/haphic build corrected_asm.fa asm.fa HiC.filtered.bam final_tours/group*.tour --corrected_ctgs corrected_ctgs.txt
  9. Generate Juicebox files for manual curation (YaHS-style)

    main

    HapHiC recommends the YaHS-style method for generating .assembly and .hic files for Juicebox curation because it avoids the need to re-align Hi-C data. This method uses a scaffolds.raw.agp file where broken contigs retain their original IDs, using coordinates for the breaks instead.

    Requirements:

    • Java and samtools must be in your $PATH.
    • For large genomes, adjust the memory settings (e.g., -Xmx64G) in the generated juicebox.sh script.

    Workflow:

    1. Run the HapHiC pipeline. It automatically generates a juicebox.sh script.
    2. Execute the script: bash juicebox.sh.
    3. Perform manual curation in Juicebox.
    4. After curation, use the haphic juicer post command to generate the final FASTA file from the reviewed assembly.
    # 1. Run the generated shell script (use bash, not sh)
    $ bash juicebox.sh
    
    # 2. After manual curation, generate the final FASTA file
    $ /path/to/HapHiC/utils/juicer post -o out_JBAT out_JBAT.review.assembly out_JBAT.liftover.agp asm.fa
  10. Verify HapHiC installation and dependencies

    main

    After installation, verify that all dependencies are correctly installed using the check command and view available commands with the -h flag.

    $ /path/to/HapHiC/haphic check
    $ /path/to/HapHiC/haphic -h
  11. Use HapHiC with hifiasm assemblies

    main

    When scaffolding haplotype-resolved assemblies from hifiasm, you can provide the GFA files produced by hifiasm (e.g., *.hap*.p_ctg.gfa for contigs or *.p_utg.gfa for unitigs) to the haphic pipeline command.

    Benefits of using --gfa:

    1. Filtering: HapHiC uses sequencing depth information from the GFA to filter potential collapsed contigs/unitigs before clustering.
    2. Haplotype Awareness: For phased contigs (*.hap*.p_ctg.gfa), HapHiC uses the phasing information to reduce or remove Hi-C connections between different haplotypes, preventing contigs from different haplotypes from clustering into the same group.

    Key Parameters:

    • --gfa: Path to the GFA file (or a comma-separated list of GFA files for multiple haplotypes).
    • --phasing_weight: Controls confidence in the hifiasm phasing.
      • 0: Completely ignores phasing information.
      • 0-1: A weight between 0 and 1 allows HapHiC to consider both hifiasm and its own phasing results (contigs from different haplotypes might cluster together).

    Note: Using --gfa to remove inter-haplotype connections may slightly decrease the overall assembly rate.

    # (1) For hifiasm unitigs (using depth to filter collapsed unitigs)
    $ /path/to/HapHiC/haphic pipeline p_utg.fa HiC.filtered.bam nchrs --gfa p_utg.gfa
    
    # (2) For hifiasm phased contigs (reduces inter-haplotype Hi-C connections)
    $ /path/to/HapHiC/haphic pipeline allhaps.fa HiC.filtered.bam nchrs --gfa "hap1.p_ctg.gfa,hap2.p_ctg.gfa"
    
    # (2b) Using --phasing_weight to control confidence (0-1 range)
    $ /path/to/HapHiC/haphic pipeline allhaps.fa HiC.filtered.bam nchrs --gfa "hap1.p_ctg.gfa,hap2.p_ctg.gfa" --phasing_weight 0
  12. Step 1: Cluster contigs using MCL

    main

    The clustering step performs preprocessing to correct assembly errors, filter short/erroneous contigs, and remove Hi-C signals between alleles. It uses the Markov Clustering (MCL) algorithm to group contigs. Unlike Agglomerative Hierarchical Clustering (AHC), MCL uses an inflation parameter to control the number of clusters (higher inflation = more clusters). HapHiC automatically tests a range of inflation values (controlled by --min_inflation, --max_inflation, and --inflation_step) and recommends the 'best' value in HapHiC_cluster.log based on the expected number of chromosomes (nchrs) and the resulting length distribution.

    Key Outputs:

    • corrected_asm.fa: Corrected contig sequences (only if assembly error correction is enabled).
    • corrected_ctgs.txt: List of corrected contig IDs.
    • full_links.pkl: Binary file of Hi-C connections between contigs.
    • HT_links.pkl: Binary file of Hi-C connections between contig ends/heads.
    • paired_links.clm: Text file of paired Hi-C connection positions.
    • inflation_*: Directories containing MCL results (mcl_inflation_*.clusters.txt) and group info (group*.txt).
    /path/to/HapHiC/haphic cluster asm.fa HiC.filtered.bam nchrs