PoseBench: Protein-Ligand Structure Prediction Benchmarking

repository·main·Indexed 42 days ago

https://github.com/bioinfomachinelearning/posebench

A comprehensive benchmarking framework for evaluating protein-ligand structure prediction methods across multiple datasets. Version 1.1.0 provides a standardized way to assess various inference methods, including integrations and documentation for tools such as Boltz, Chai-1, and DiffDock.

Tokens
70.5K
Snippets
154
Records
203
Agent score
47%

What's inside PoseBench

  1. Overview of Evolutionary Scale Modeling (ESM) models

    main

    The ESM repository provides Transformer-based protein language models developed by Meta FAIR. These models are designed for various protein-related tasks including structure prediction, functional property prediction, and sequence design.

    Key model families include:

    • ESM-2: State-of-the-art (SOTA) general-purpose protein language models used for predicting structure, function, and other properties from single sequences.
    • ESMFold: An end-to-end model that generates accurate 3D protein structures directly from a sequence by leveraging ESM-2.
    • MSA Transformer (ESM-MSA-1b): A model designed to extract embeddings from Multiple Sequence Alignments (MSAs), enabling high-performance structure inference.
    • ESM-1v: A specialized model for zero-shot prediction of the functional effects of sequence variations (variant effects).
    • ESM-IF1: An inverse folding model used for fixed-backbone sequence design or predicting the effects of variations on a given structure.
  2. Overview of the ESM Metagenomic Atlas

    main

    The ESM Metagenomic Atlas is a repository containing over 600 million metagenomic protein structures predicted using the ESMFold model (esm.pretrained.esmfold_v0()).

    It is divided into two main offerings:

    1. High quality MGnify30 structures (~1TB): A subset built from a 30% sequence identity clustering of MGnify90. The best structure from each cluster is selected based on the score pTM * pLDDT. Structures are filtered to have pTM > 0.7 and pLDDT > 0.7.
    2. Full database (~15TB): The complete set of PDB structures, stratified by pTM and pLDDT scores to allow for selective downloading based on quality requirements.
  3. Overview of SE(3)-Transformers For PyTorch

    main
    This repository provides scripts and recipes for training the SE(3)-Transformer model to achieve state-of-the-art accuracy. The implementation is maintained and tested by NVIDIA. It includes details on model architecture, default configurations, feature support, mixed precision training, and performance benchmarking for both training and inference.
  4. What is the SE(3)-Transformer?

    main

    The SE(3)-Transformer is a Graph Neural Network (GNN) designed for processing 3D points and graphs using a variant of self-attention. It is equivariant under continuous 3D roto-translations (the SE(3) group), meaning that if the input (graphs or point sets) undergoes a rigid transformation (rotation or translation), the model's outputs either remain invariant or transform accordingly.

    This implementation is optimized for high throughput and low memory consumption compared to the official implementation, and it supports:

    • Multi-GPU training and inference (DDP).
    • Mixed precision training (AMP).
    • Edge features of any degree (not just scalar degree 0).
    • Automatic downloading of the QM9 dataset from DGL.
    • Computation of spherical harmonics and Clebsch–Gordan coefficients via the e3nn library.
  5. Understand the Boltz prediction output directory structure

    main

    After running a prediction, results are organized in an output directory (out_dir/).

    • lightning_logs/: Logs from training or evaluation.
    • predictions/: Contains subdirectories for each input file. Each input subdirectory contains:
      • [input_file]_model_[N].cif: Predicted structure in CIF format (includes per-token pLDDT scores).
      • confidence_[input_file]_model_0.json: Aggregated confidence scores.
      • affinity_[input_file].json: Affinity scores.
      • pae_[input_file]_model_0.npz: Predicted PAE scores for every pair of tokens.
      • pde_[input_file]_model_0.npz: Predicted PDE scores for every pair of tokens.
      • plddt_[input_file]_model_0.npz: Predicted pLDDT scores for every token.
    • processed/: Contains the processed input files used during inference.
  6. Understand Binding Affinity Prediction outputs

    main

    Boltz-2 provides two distinct types of affinity predictions in its output. They should be used for different stages of molecular design:

    1. affinity_probability_binary: A value ranging from 0 to 1 representing the predicted probability that the ligand is a binder. Use this for hit-discovery (detecting binders from decoys).
    2. affinity_pred_value: A value reported as log(IC50) (derived from IC50 measured in μM). Use this for ligand optimization (hit-to-lead and lead-optimization) to measure how specific affinity changes with molecular modifications.
  7. Interpret DiffDock confidence scores

    main

    DiffDock outputs a confidence score ($c$) for the top pose. This score measures the quality of the predicted 3D structure, not the binding affinity.

    General Guidelines for Score $c$:

    • $c > 0$: High confidence
    • $-1.5 < c < 0$: Moderate confidence
    • $c < -1.5$: Low confidence

    Important Caveats:

    • These intervals assume the complex is similar to the training set (e.g., a drug-like molecule bound to a medium-sized protein in a conformation similar to the bound one).
    • If you are working with large ligands, large protein complexes, or unbound protein conformations, you should shift these intervals down (expect lower scores).
    • The score is not a direct measure of binding affinity. For affinity prediction, it is recommended to combine DiffDock predictions with tools like GNINA or MM/GBSA, ideally after relaxing the predicted structure with a force field.
  8. How the SE(3)-Transformer architecture works

    main

    The model follows a graph-to-scalar architecture (though it can be adapted to graph-to-graph):

    1. Stacked Layers: Consists of equivariant graph self-attention and equivariant normalization layers.
    2. Tensor Field Network (TFN): A TFN convolution is applied to obtain invariant features.
    3. Pooling: Graph pooling (mean or max over nodes) is applied to these features.
    4. MLP: A final Multi-Layer Perceptron (MLP) processes the pooled features to produce scalar predictions.

    To obtain a graph-to-graph network, the pooling step can be removed. To output different feature types (e.g., 3D vectors), the final TFN can be modified.

  9. Understand the Chai Discovery Community License

    main

    The Chai-1 Model software code and model weights (the "AI Model") are provided under the Chai Discovery Community License. This license is strictly limited to Non-Commercial Purposes.

    Key Restrictions:

    • No Commercial Use: You cannot use the AI Model, Outputs, or Derivative Works for any activity intended for commercial advantage or monetary compensation. This excludes Commercial Entities (entities engaged in commercial activity, excluding universities, non-profits, and government bodies).
    • No Drug Discovery: Use of the AI Model, Outputs, or Derivative Works is strictly prohibited in connection with drug development or discovery. This includes, but is not limited to: drug/medication/therapeutic discovery, target/hit/lead identification, drug candidate selection, or lead optimization.
    • No Service Offerings: You may not use the AI Model or its outputs to provide services to third parties (e.g., a hosted service providing model outputs), even if no fee is charged.
    • Model Training Restrictions: Per the Acceptable Use Policy (AUP), the AI Model and its outputs may not be used to train, optimize, or improve any AI/ML models with more than 10,000 trainable parameters, or technology for protein structure prediction or protein, drug, or enzyme design.