Ensembl VEP

repository·release/116·Indexed 20 days ago

https://github.com/ensembl/ensembl-vep

A suite of tools for genomic variant analysis featuring the Variant Effect Predictor (VEP) for functional effect prediction, Haplosaurus (haplo) for haplotype sequence prediction, and a Variant Recoder for translating between variant encodings (VCF, variant ID, HGVS). Includes a Nextflow pipeline to parallelize VEP processing by splitting input data into bins.

Tokens
3.5K
Snippets
12
Records
18
Agent score
69%

What's inside ensembl-vep

  1. How Haplosaurus (`haplo`) works

    release/116

    Haplosaurus (haplo) is a tool that implements the functionality of the Ensembl transcript haplotypes view. Unlike VEP, which analyzes variants independently, haplo considers the combined effect of all variant alleles across a transcript by constructing haplotype sequences.

    Key capabilities:

    • Takes phased genotypes from a VCF and constructs pairs of haplotype sequences (CDS and protein).
    • Aligns and compares variant haplotype sequences to the reference.
    • Constructs HGVS-like names representing differences to the reference.
    • Uses VEP caches, Ensembl databases, GFF, and GTF files as transcript data sources.

    Requirements:

    • Input must be a VCF containing phased genotype data.
    • The VCF file must be sorted by chromosome and genomic position.
    ./haplo -i input.vcf -o out.txt -cache
  2. Use Docker or Conda for ensembl-vep

    release/116

    Docker

    A Docker image is available on DockerHub at ensemblorg/ensembl-vep. Refer to the official documentation for specific installation instructions.

    Conda

    An Ensembl VEP Conda package exists, but it is not produced or maintained by the Ensembl VEP team. Use it at your own risk, as the team cannot provide support for issues related to the Conda installation.

  3. Use Variant Recoder to translate variant encodings

    release/116

    The variant_recoder tool translates between different variant encodings (VCF, variant ID, HGVS). It supports genomic, transcript, and protein HGVS, as well as RefSeq and LRG sequences.

    Important Notes:

    • It requires database access for identifier lookup and cannot be used in offline mode.
    • It requires the JSON Perl module.
    • Output is in JSON format.

    Usage Examples:

    • For a single variant string: ./variant_recoder --id [input_data_string]
    • For an input file (one variant per line): ./variant_recoder -i [input_file] --species [species]
    ./variant_recoder --id "AGT:p.Met259Thr" --pretty
    ./variant_recoder -i [input_file] --species [species]
  4. Install optional CPAN modules for ensembl-vep

    release/116

    While the core installation handles basic requirements, several optional CPAN modules provide significant functionality or performance improvements. It is recommended to use cpanm to install them:

    • DBD::mysql: Required for database access (using --database or --cache without --offline).
    • Set::IntervalTree: Required for Haplosaurus and improves VEP speed.
    • JSON: Required for writing JSON output.
    • PerlIO::gzip: Enables faster compressed file parsing.
    • Bio::DB::BigFile: Required for reading custom annotation data from BigWig files.
    # Example using cpanm (recommended)
    cpanm DBD::mysql Set::IntervalTree JSON PerlIO::gzip Bio::DB::BigFile
  5. Install `bioperl-ext` for faster `haplo` analysis

    release/116

    To speed up haplo analysis (especially for long transcripts with indels), you can install the bioperl-ext package. Note that this package is no longer maintained and requires manual installation. The following steps install it to $HOME/perl5.

    Warning: You may need to adjust the export PERL5LIB command to match your specific architecture and Perl version.

    git clone https://github.com/bioperl/bioperl-ext.git
    cd bioperl-ext/Bio/Ext/Align/
    perl -pi -e"s|(cd libs.+)CFLAGS=\\'|\$1CFLAGS=\\'-fPIC |" Makefile.PL
    perl Makefile.PL PREFIX=~/perl5
    make
    make install
    cd -
    export PERL5LIB=${PERL5LIB}:${HOME}/perl5/lib/x86_64-linux-gnu/perl/5.22.1/
  6. Install ensembl-vep

    release/116

    To install the ensembl-vep package, you need gcc, g++, make, and Perl (>=5.22 recommended). You must also have the Perl libraries Archive::Zip and DBI installed.

    Follow these steps to install from source:

    1. Clone the repository.
    2. Run the INSTALL.pl script.

    You can re-run INSTALL.pl to check for updates to the package and its dependencies.

    git clone https://github.com/Ensembl/ensembl-vep.git
    cd ensembl-vep
    perl INSTALL.pl
  7. Install and set up the Nextflow VEP pipeline

    release/116

    The Nextflow VEP pipeline parallelizes VEP by splitting input data into bins, running VEP on those bins in parallel, and merging the results.

    Requirements

    • Nextflow (tested on version 21.10.0)

    Setup Steps

    1. Configure VEP: Create a vep.ini file by copying the template:

      cp vep_config/vep.ini.template vep_config/vep.ini

      Note: Always use absolute paths in vep.ini when specifying directories.

    2. Nextflow Configuration: The nextflow.config file contains default pipeline options. You can modify this file directly or override options via the command line.

    3. VEP Cache (Optional): For faster performance, use a VEP cache instead of a database. Download and extract the cache from the Ensembl FTP and ensure the absolute path to the cache directory is included in your vep.ini file.

    cp vep_config/vep.ini.template vep_config/vep.ini
  8. Set up MySQL for running DB tests

    release/116

    To run database tests, you must have write access to a MySQL server. You need to configure the test database parameters by creating a configuration file from the provided example.

    1. Copy MultiTestDB.conf.example to MultiTestDB.conf.
    2. Edit MultiTestDB.conf to set the required connection and database parameters.
    cp MultiTestDB.conf.example MultiTestDB.conf
  9. Generate test databases using clone_core_database.pl

    release/116

    You can generate test databases by cloning a core database from the Ensembl server using the clone_core_database.pl script. This requires a JSON configuration file specifying the regions to be used.

    Note on species naming: To prevent collisions with existing databases on your host, the script is configured to rename the species to homo_vepiens. After cloning, you must rename the directory in t/test-genome-dbs and update the metadata in t/test-genome-dbs/homo_vepiens/core/meta.txt to reflect the changes.

    perl ../../ensembl-test/scripts/clone_core_database.pl -host ensembldb.ensembl.org \\n-user anonymous -dbname homo_sapiens_core_84_38 -json test_core_region_config.json \\n-directory t/test-genome-dbs -dest_host [host] -dest_user [user] -dest_pass [pass]
  10. Troubleshoot known VEP bugs and workarounds

    release/116

    Several known issues exist in VEP. Key workarounds include:

    • Multi-allelic variants with --hgvs and --no_stats: This combination can cause crashes or incorrect positions. Workaround: Separate multi-allelic variants into individual input lines, or avoid combining these two specific flags.
    • MAF (Minor Allele Frequency) filtering: Using --freq_freq, --freq_gt_lt, or --freq_filter can incorrectly include/exclude indels. Workaround: Do not filter during the VEP run; instead, use the filter_vep script on the output files.
    • --shift_3prime 1 with custom annotations: Exact allele matching in custom lookups (e.g., --custom <file>,...,vcf,exact,...) may fail. Workaround: Run VEP in two passes. Pass 1 with --shift_3prime 1 to get consequence/HGVS, and Pass 2 with --shift_3prime 0 to get custom annotations, then merge the results using bcftools annotate or a custom script.
    • Non-deterministic annotation ordering: In Perl >=5.18, the CSQ field order may vary between runs (especially with --pick_allele_gene). Workaround: Set the environment variables PERL_HASH_SEED=0 and PERL_PERTURB_KEYS=0 to ensure deterministic ordering.