genomepy

genomepy is a Python and command-line tool designed for managing genomic data. Its primary functions include:

• Finding: Searching for genomes and gene annotations.
• Downloading: Retrieving genomic data from various providers.
• Preprocessing: Preparing genome and annotation files for use.
• Handling: Managing and interacting with genomes and gene annotations.

Plugins are optional post-installation steps. You can enable them for existing installations by rerunning the install command (use --force to overwrite existing files).

Available Plugins:

• blacklist: Downloads genome blacklists (available for UCSC model organisms, human, and mouse).
• Aligner Indexes: Creates indexes for bwa, bowtie2, gmap, hisat2, minimap2, and star.
  • Note: The aligner software itself must be installed on your system (e.g., via conda).
  • Note: Splice-aware indexing (for STAR and HISAT2) is automatic if --annotation was used during installation.

Commands:

• genomepy plugin list: List enabled plugins.
• genomepy plugin enable <plugin_name>: Enable a plugin.
• genomepy install --threads <N>: Set threads for index creation.

$ genomepy plugin enable bwa star
$ genomepy install xenTro9 --threads 16

If a genome cannot be found via the standard search (due to naming inconsistencies on the provider side), you can manually install it by providing the direct URL to the genome file and the annotation file.

# Use the -p url flag to indicate the input is a URL
# Use --url-to-annotation to specify the annotation source
genomepy install GENOME_URL -p url --url-to-annotation ANNOTATION_URL

To find available genomes or gene annotations, use genomepy.search. This function searches through one or all available providers. To optimize speed, you can specify a specific provider.

Before searching, you can inspect available providers using:

• genomepy.list_providers(): Lists all available providers.
• genomepy.list_online_providers(): Lists only online providers.

If you want to see every genome available without a specific query, use genomepy.list_available_genomes(), but be aware this can return a very large list.

import genomepy

# Search for a specific organism
results = genomepy.search('homo sapiens')

# List providers to narrow down search
providers = genomepy.list_online_providers()
results = genomepy.search('homo sapiens', provider=providers[0])

Genomepy uses a YAML configuration file to manage settings. You can generate a new configuration file using the config generate command, or locate the path to your existing configuration file using the config file command.

# Generate a new config file
genomepy config generate

# Find the location of your current config file
genomepy config file

You can quickly find and download genomic data using the following workflow:

1. Search for a genome by species name:

   genomepy search zebrafish

   This returns a table containing the name, provider, accession, tax_id, annotation status, species, and other_info. Use the value in the name column for the installation step.

2. Install the genome and its annotations:

   genomepy install --annotation <NAME> --provider <PROVIDER>

   Example: genomepy install --annotation GRCz11 --provider ensembl

By default, genomes are installed to ~/.local/share/genomes/.

# 1. Search
gemomepy search zebrafish

# 2. Install (using name from search results)
gemomepy install --annotation GRCz11 --provider ensembl

genomepy requires Python 3.9+. You can install it using Bioconda, pip, or by cloning the Git repository.

Bioconda (Recommended)

conda install -c conda-forge -c bioconda 'genomepy>=0.16'

Pip

pip install genomepy

Note: Pip installations require manual installation of additional dependencies to be available in your PATH.

Git

git clone https://github.com/vanheeringen-lab/genomepy.git
conda env create -n genomepy -f genomepy/environment.yml
conda activate genomepy
pip install -e genomepy

Configuration

Genomes and annotations are stored in a genomes directory by default. You can change this default location using genomepy.manage_config. Use genomepy.manage_config() to find and inspect your current configuration.

Plugins

For specialized tasks like downloading sequence blacklists or creating aligner indices, use genomepy.manage_plugins(). These are treated as extensions to the genome installation process.

If you install via pip, you must manually install and add the following to your PATH to enable specific features:

For bgzipped genome support

• pysam

For gene annotation features

You need the following UCSC binaries:

• genePredToBed
• genePredToGtf
• bedToGenePred
• gtfToGenePred
• gff3ToGenePred

Binaries can be found at: http://hgdownload.cse.ucsc.edu/admin/exe/

Use genomepy.install_genome to download and set up a genome. The default parameters are optimized for sequence alignment and gene counting.

Key behaviors:

• It will not overwrite existing files unless explicitly specified.
• You can check what you have already downloaded using genomepy.list_installed_genomes().
• If you need to add additional components (like sequence blacklists or aligner indices) after an initial installation, you can rerun genomepy.install_genome; genomepy will only execute the new parts.

import genomepy

# Install a genome
genomepy.install_genome('hg38')

# List what is currently installed locally
installed = genomepy.list_installed_genomes()

When providing a list of regions to as_seqdict(), you can bypass the need for a global genome argument by embedding the genome name directly into each region string. This is useful when working with multiple genomes in a single list.

Format: genome_name@chromosome:start-end

Example: "hg38@chr1:1000-2000"