Foldcomp

repository·master·Indexed 42 days ago

https://github.com/steineggerlab/foldcomp

A library and file format for the efficient compression and indexing of large protein structure sets. It uses a compact binary format (FCZ) to reduce storage to approximately 13 bytes per residue by encoding backbone and side-chain torsion angles. Foldcomp provides a CLI for compression, decompression, and extraction, a Python API for programmatic access and database management, and a C++ implementation including the AminoAcid and AtomCoordinate classes.

Tokens
12.2K
Snippets
40
Records
55
Agent score
52%

What's inside Foldcomp

  1. Use custom streams with microtar

    master

    Instead of reading from or writing to a file, you can manually initialize the mtar_t struct to work with custom streams.

    Steps to use a custom stream:

    1. Manually initialize the mtar_t struct.
    2. Set the stream pointer.
    3. Provide the required callback functions.
    4. Ensure all unused fields are zero-initialized.
    5. Do not call mtar_open if you have initialized the struct manually.

    Required Callbacks for Reading

    NameArgumentsDescription
    readmtar_t *tar, void *data, unsigned sizeRead data from the stream
    seekmtar_t *tar, unsigned posSet the position indicator
    closemtar_t *tarClose the stream

    Required Callbacks for Writing

    NameArgumentsDescription
    writemtar_t *tar, const void *data, unsigned sizeWrite data to the stream

    All callbacks must return MTAR_ESUCCESS on success or a negative integer on error.

  2. Install Foldcomp

    master

    You can install Foldcomp as a Python package via pip, or download static binaries for various operating systems.

    Python Package

    pip install foldcomp

    Static Binaries

    • Linux (x86_64): wget https://mmseqs.com/foldcomp/foldcomp-linux-x86_64.tar.gz
    • Linux (ARM64): wget https://mmseqs.com/foldcomp/foldcomp-linux-arm64.tar.gz
    • macOS (Universal): wget https://mmseqs.com/foldcomp/foldcomp-macos-universal.tar.gz
    • Windows (x64): wget https://mmseqs.com/foldcomp/foldcomp-windows-x64.zip
    pip install foldcomp
  3. Download prebuilt Foldcomp databases via Python

    master

    Foldcomp provides a Python helper to download large prebuilt databases of predicted protein structures. Use foldcomp.setup('database_name') to initiate the download.

    Available Databases

    • ESMAtlas:
      • Full: foldcomp.setup('esmatlas')
      • v2023_02: foldcomp.setup('esmatlas_v2023_02')
      • High-quality: foldcomp.setup('highquality_clust30')
    • AlphaFoldDB Uniprot:
      • v6: foldcomp.setup('afdb_uniprot_v6')
      • v4: foldcomp.setup('afdb_uniprot_v4')
    • AlphaFoldDB Swiss-Prot:
      • foldcomp.setup('afdb_swissprot_v4')
    • AlphaFoldDB Model Organisms:
      • foldcomp.setup('h_sapiens')
      • foldcomp.setup('a_thaliana'), foldcomp.setup('c_albicans'), foldcomp.setup('c_elegans'), foldcomp.setup('d_discoideum'), foldcomp.setup('d_melanogaster'), foldcomp.setup('d_rerio'), foldcomp.setup('e_coli'), foldcomp.setup('g_max'), foldcomp.setup('m_jannaschii'), foldcomp.setup('m_musculus'), foldcomp.setup('o_sativa'), foldcomp.setup('r_norvegicus'), foldcomp.setup('s_cerevisiae'), foldcomp.setup('s_pombe'), foldcomp.setup('z_mays')
    • AlphaFoldDB Cluster Representatives:
      • foldcomp.setup('afdb_rep_v4')
      • foldcomp.setup('afdb_rep_dark_v4')
    import foldcomp; foldcomp.setup('afdb_swissprot_v4')
  4. Set up Foldcomp databases

    master

    Foldcomp provides a way to handle large protein structure databases (like AlphaFold DB) by storing them in a single compressed file.

    Use foldcomp.setup(db_name) in a local environment to download and prepare a database. If working in an asynchronous environment like Google Colab, use await foldcomp.setup_async(db_name).

    Example database name: afdb_swissprot_v4.

    # In local python environment:
    foldcomp.setup("afdb_swissprot_v4")
    
    # In Google Colab (async):
    await foldcomp.setup_async("afdb_swissprot_v4")
    # use the built-in downloader
    await foldcomp.setup_async("afdb_swissprot_v4")
  5. Use the foldcomp CLI for protein structure compression and decompression

    master

    The foldcomp executable provides a command-line interface for lossy compression of protein structures. It encodes torsion angles to save space and can reconstruct 3D coordinates from the encoded data.

    Basic Usage

    • Compress a single file: foldcomp compress <input.pdb|input.cif> <output.fcz>
    • Decompress a file: foldcomp decompress <input.fcz> <output.pdb>

    Batch Operations

    Foldcomp supports processing directories, tar files, and databases:

    • Compress a directory/tar: foldcomp compress [-t threads] <dir|tar(.gz)> <output_dir|tar|db>
    • Decompress a directory/tar: foldcomp decompress [-t threads] <dir|tar(.gz)|db> <output_dir|tar>

    Extraction and Checking

    • Extract metadata: Use extract to pull pLDDT scores or amino acid sequences from .fcz files.
    • Check integrity: Use check to verify the validity of .fcz files.
    # Compress a PDB file
    foldcomp compress input.pdb output.fcz
    
    # Decompress an FCZ file
    foldcomp decompress output.fcz output.pdb
    
    # Compress a directory using 4 threads
    foldcomp compress -t 4 ./pdb_dir ./output_dir
  6. Handle FCZ files and databases with the Python API

    master

    The Foldcomp Python API allows for programmatic decompression and data extraction from compressed FCZ files and databases.

    Decompressing a single FCZ file

    Use foldcomp.decompress(fcz_binary) to get the filename and the PDB content as a string.

    Extracting data from an FCZ file

    Use foldcomp.get_data(fcz_binary) (or foldcomp.get_data(pdb_path)) to retrieve a dictionary containing:

    • phi: phi angles (C-N-CA-C)
    • psi: psi angles (N-CA-C-N)
    • omega: omega angles (CA-C-N-CA)
    • torsion_angles: backbone torsion angles (phi + psi + omega)
    • bond_angles: backbone bond angles
    • residues: amino acid residues as a string
    • coordinates: backbone coordinates

    Iterating over a database

    Use foldcomp.open(path, ids=[...]) to iterate over specific entries in a Foldcomp database.

    import foldcomp
    
    # Decompressing a single FCZ file
    with open("test/compressed.fcz", "rb") as fcz:
        fcz_binary = fcz.read()
        (name, pdb) = foldcomp.decompress(fcz_binary)
        with open(name, "w") as pdb_file:
            pdb_file.write(pdb)
    
    # Getting data as a dictionary
    data_dict = foldcomp.get_data(fcz_binary)
    print(data_dict["phi"])
    
    # Iterating over a database
    ids = ["d1asha_", "d1it2a_"]
    with foldcomp.open("test/example_db", ids=ids) as db:
        for (name, pdb) in db:
            with open(name + ".pdb", "w") as pdb_file:
                pdb_file.write(pdb)
  7. Calculate RMSD between two structures with foldcomp

    master

    Use the rmsd mode to compare the structural similarity between two PDB or mmCIF files. The tool calculates both the backbone RMSD and the all-atom RMSD.

    Usage: foldcomp rmsd <pdb1|cif1> <pdb2|cif2>

    Output Format: pdb1 pdb2 backbone_atoms_count total_atoms_count backbone_rmsd all_atom_rmsd

    foldcomp rmsd input1.pdb input2.pdb