DOCSEARCHbyAlgolia
Submit Docs

Evolutionary Scale Modeling (ESM)

repository·main·Indexed 26 days ago

https://github.com/facebookresearch/esm

A suite of Transformer-based protein language models from Meta FAIR. ESM provides tools for protein structure prediction (ESMFold), sequence design (ESM-IF1), variant effect prediction (ESM-1v), and general-purpose protein property modeling (ESM-2). The repository includes implementations for inverse folding, fixed backbone design, free generation, and a protein programming language for generative protein design.

Tokens
18.3K
Snippets
50
Records
71
Agent score
87%

What's inside ESM

  1. Overview of Evolutionary Scale Modeling (ESM)

    main

    Evolutionary Scale Modeling (ESM) is a collection of Transformer-based protein language models developed by Meta's Fundamental AI Research (FAIR) team. The repository provides pre-trained weights and code for several state-of-the-art models used to predict protein structure, function, and properties directly from sequences.

    Key capabilities include:

    • Structure Prediction: Using ESM-2 or ESMFold to predict 3D protein structures.
    • Variant Effect Prediction: Using ESM-1v for zero-shot prediction of functional effects from sequence variations.
    • Inverse Folding: Using ESM-IF1 to design sequences for specific protein structures.
    • MSA-based Inference: Using MSA Transformer for extracting embeddings from Multiple Sequence Alignments (MSAs).

  2. Access the ESM Metagenomic Atlas

    main

    The ESM Metagenomic Atlas provides resources for protein research, including sequence folding and structural/sequence searching.

    • Web Interface: Visit esmatlas.com to fold sequences using ESMFold or search the Atlas by structure or sequence.
    • API: Programmatic access is available via the Atlas API.
    • Bulk Downloads: Instructions for bulk downloading Atlas resources are located in scripts/atlas/README.md.
    • Foldseek: Use Foldseek to perform searches against the Atlas without length limitations.

  3. Access ESM Metagenomic Atlas metadata

    main

    The ESM Metagenomic Atlas provides metadata for over 700 million predicted protein structures. You can download the metadata as a .sqlite or .parquet file to act as an entry point for the database.

    To load the metadata using pandas:

    import pandas as pd
# Download from: https://dl.fbaipublicfiles.com/esmatlas/v2023_02/metadata-rc2.parquet
df = pd.read_parquet('metadata.parquet')

    Metadata Schema (10 columns):

    • id: MGnify ID
    • ptm: Predicted TM score
    • plddt: Predicted average lDDT
    • num_conf: Number of residues with plddt > 0.7
    • len: Total residues in the protein
    • is_fragment: Boolean indicating if the sequence is a fragment in the MGnify90 database
    • sequenceChecksum: CRC64 hash of the sequence (useful for cheap lookups)
    • esmfold_version: Version of ESMFold used (esm.pretrained.esmfold_v0 or v1)
    • atlas_version: The Atlas version where this structure first appeared
    • sequence_dbs: Comma-separated list of metagenomic source databases (e.g., MGnify90_2022_05,MGnify90_2023_02)

    Note: Proteins longer than 1280 residues were not folded. To filter for records that have valid structure predictions, use: df[~df.plddt.isna()].

  4. Setup environment for ESM-IF1

    main

    It is highly recommended to use a fresh conda environment to avoid CUDA compatibility issues between pytorch and pytorch-geometric.

    Follow these steps to set up the environment:

    1. Create a new conda environment with Python 3.9.
    2. Install pytorch and cudatoolkit=11.3.
    3. Install pyg (PyTorch Geometric).
    4. Install biotite and the esm repository via pip.
    conda create -n inverse python=3.9
conda activate inverse
conda install pytorch cudatoolkit=11.3 -c pytorch
conda install pyg -c pyg -c conda-forge
conda install pip
pip install biotite
pip install git+https://github.com/facebookresearch/esm.git

  5. Use the ESMStructuralSplitDataset

    main

    The ESMStructuralSplitDataset is a five-fold cross-validation dataset of protein domain structures used to measure representation generalization across family, superfamily, and fold levels.

    Upon initialization, it automatically downloads:

    • splits: train/valid splits.
    • pkl: objects containing sequence, SSP labels, distance map, and 3d coordinates.
    • msas: a3m files containing MSA for each domain.

    A Jupyter notebook tutorial is available at examples/esm_structural_dataset.ipynb for loading and indexing this dataset.

  6. Score sequences using CLI

    main

    Use the score_log_likelihoods.py script to calculate conditional log-likelihoods for sequences relative to a structure. The output is a CSV containing average log-likelihoods per sequence.

    Key Arguments:

    • --chain: The target chain to score.
    • --multichain-backbone: Use the entire complex backbone for conditioning instead of just the target chain.

    Example Command:

    python score_log_likelihoods.py data/5YH2.pdb \
    data/5YH2_mutated_seqs.fasta --chain C \
    --outpath output/5YH2_mutated_seqs_scores.csv

  7. Install ESMFold with dependencies

    main

    To use the ESMFold model, ensure you are using Python <= 3.9 and have PyTorch installed. Installing with the [esmfold] extra will automatically attempt to install OpenFold dependencies.

    Note: OpenFold installation requires nvcc. If installation fails, verify that nvcc is available and that a CUDA-compatible version of PyTorch is installed.

    Manual dependency installation for OpenFold may be required if the pip extra fails:

    1. Install dllogger from NVIDIA.
    2. Install openfold from A-Lab.

    Alternatively, you can use the provided Conda environment: conda env create -f environment.yml

    pip install "fair-esm[esmfold]"
# OpenFold and its remaining dependency
pip install 'dllogger @ git+https://github.com/NVIDIA/dllogger.git'
pip install 'openfold @ git+https://github.com/aqlaboratory/openfold.git@4b41059694619831a7db195b7e0988fc4ff3a307'
    conda env create -f environment.yml

  9. Sample sequence designs using CLI

    main

    Use the sample_sequences.py script to generate protein sequences from a structure in .pdb or .cif format.

    Key Arguments:

    • --chain: The specific chain to design sequences for.
    • --temperature: Controls sampling sharpness. Default is 1. Use low temperatures (e.g., 1e-6) to optimize for native sequence recovery.
    • --num-samples: Number of sequences to generate.
    • --outpath: Path to save the resulting FASTA file.
    • --multichain-backbone: If provided, the model uses the entire complex backbone as input for conditioning, rather than just the target chain.

    Note: Check sampled sequences for long repeats (e.g., EEEEEEEE) and filter them out if necessary.

    # Single-chain design
python sample_sequences.py data/5YH2.pdb \
    --chain C --temperature 1 --num-samples 3 \
    --outpath output/sampled_sequences.fasta

# Multi-chain conditioning design
python sample_sequences.py data/5YH2.pdb \
    --chain C --temperature 1 --num-samples 3 \
    --outpath output/sampled_sequences_multichain.fasta \
    --multichain-backbone

  11. Generate MSAs using hhblits

    main

    You can generate Multiple Sequence Alignments (MSAs) in .a3m format similar to those used for MSA Transformer pre-training using hhblits.

    Use the following command structure: hhblits -i UniRef50_$id.fas -oa3m UniRef50_$id.a3m -n 3 -d /uniclust30_2017_10/uniclust30_2017_10

    hhblits -i UniRef50_$id.fas -oa3m UniRef50_$id.a3m -n 3 -d /uniclust30_2017_10/uniclust30_2017_10

  12. Bulk download ESM Metagenomic Atlas structures

    main

    The Atlas database is partitioned by pTM and pLDDT scores to allow targeted downloads. You can download specific bundles of structures, Foldseek databases, or ESM2 embeddings.

    Available Data Types:

    • Bulk Predicted structures: Found in {v0,v2023_02}/full/tarballs/.
    • Foldseek DBs: Found in {v0,v2023_02}/full/foldseekdb/.
    • Bulk Embeddings: Found in {v0,v2023_02}/full/esm2_embeddings/.

    Download Method: It is recommended to use aria2c or s5cmd for downloading large files. To download structures using aria2c from a provided URL list file:

    aria2c --dir=/path/to/download/to --input-file=url-file-provided.txt