DOCSEARCHbyAlgolia
Submit Docs

QCArchive

repository·main·Indexed 41 days ago

https://github.com/molssi/qcfractal

A scalable distributed compute and database platform for managing, compiling, and sharing large-scale quantum chemistry data and computations. The platform consists of several components: qcfractal (the core server and API), qcportal (the Python client), qcfractalcompute (compute workers), and qcarchivetesting (testing helpers).

Tokens
91.8K
Snippets
270
Records
364
Agent score
49%

What's inside QCArchive

  1. What is QCArchive and its architecture

    main

    QCArchive is a platform designed for running, storing, and managing large-scale quantum chemistry calculations (thousands to millions of computations). It is built to handle the complexities of different QM codes, distributed resources, and error handling at scale.

    The architecture consists of three main components:

    1. QCFractal (Server): The central component responsible for storing and managing data. It exposes a JSON-based web API, allowing access via any language capable of making web requests.
    2. QCPortal (Client): A Python-native client used to submit and manage calculations. This is the primary way for users to interact with the server.
    3. QCFractalCompute (Workers): The component responsible for actually executing the calculations. These are typically set up on a supercomputing cluster.

    Typical Deployment Pattern:

    • Server: Run on a dedicated machine.
    • Client: Run on the user's local machine.
    • Workers: Set up on a supercomputing cluster, often with a process on the head node to coordinate job submission and monitoring.

  2. Overview of the QCArchive architecture

    main

    QCArchive is a platform for managing and executing large-scale quantum chemistry calculations. The architecture is composed of three primary components that work together to handle data from thousands to millions of computations:

    1. QCFractal: The central server that manages the database for storing, sharing, retrieving, and analyzing results.
    2. QCPortal: The client interface used by end-users to interact with the system.
    3. QCFractalCompute: The computational workers responsible for executing the actual quantum chemistry calculations.

    This modular design ensures a streamlined workflow for managing high-volume computational chemistry data.

  3. Overview of QCArchive components

    main

    QCArchive is organized as a monorepo containing several interdependent Python packages. Understanding these components is key to using the platform:

    • qcfractal: The core server, providing the database and the web API.
    • qcportal: The Python client used to interact with the qcfractal server.
    • qcfractalcompute: Worker components designed to be deployed to run actual quantum chemistry computations.
    • qcarchivetesting: A suite of helpers and pytest harnesses used for testing the various QCArchive components.

  4. What is the QCFractal Snowflake?

    main

    The QCFractal Snowflake is a self-contained server and compute manager designed for experimentation and testing. It allows you to use QCFractal without setting up a standalone server.

    Important Caveats:

    • Data Volatility: The database is stored in a temporary directory and is permanently deleted when the snowflake is destructed. It is not intended for permanent data storage.
    • Security: Security is not enabled by default.
    • Compute Limits: You are generally limited to a few compute workers, which run on the same machine as the server.
    • Dependencies: The host computer must have Postgres installed, and any programs required for computations (e.g., Psi4) must be available in the environment where the snowflake is running.

  5. Understand Nudged Elastic Band (NEB) calculations

    main

    A Nudged Elastic Band (NEB) calculation finds the minimum energy path between two molecular endpoints (e.g., a reactant and a product). The path is represented as a chain of ordered molecular geometries called images.

    During the calculation, gradients are computed for every image, and spring forces are applied to maintain spacing between images. The highest-energy image in the final converged chain is considered the guessed transition state.

    NEB calculations can optionally include:

    • Endpoint optimization (optimize_endpoints): Optimizing the two endpoints of the chain before the NEB starts.
    • Transition-state refinement (optimize_ts): Refining the guessed transition state into a true first-order saddle point by computing its Hessian and running a transition-state optimization.

  6. How reaction records work

    main

    A reaction record computes a linear combination of molecular energies (e.g., reaction energy, binding energy, or atomization energy).

    A reaction is defined by its stoichiometry: a list of (coefficient, molecule) pairs. The total energy is the sum of each molecule's energy multiplied by its coefficient. Coefficients are typically negative for reactants and positive for products.

    Reaction records include all fields of a base record, plus:

    • specification: The programs and levels of theory used.
    • total_energy: The computed reaction energy in hartrees (None until complete).
    • components: A list of ReactionComponent objects, one per molecule in the stoichiometry. Each component contains the molecule, molecule_id, coefficient, and links to the singlepoint_record or optimization_record used.
    # Example stoichiometry for the dissociation of a water dimer:
[(-1.0, dimer), (2.0, monomer)]

  7. Configure default compute settings for a project

    main
    When creating a project, you can set default_compute_tag and default_compute_priority. These settings are automatically inherited by any records or datasets created within that project, reducing the need to specify them manually for every call. These can still be overridden at the individual record or dataset level. Note that compute tags are lowercased by the server.

  8. Understand QCArchive record and computation types

    main

    QCArchive supports several distinct computation types, each represented by a specific record object. While all record objects share a common set of properties and methods defined in the Base record, each type provides unique features tailored to the specific scientific computation performed.

    Available record types include:

    • Base record: Common features shared across all records.
    • Singlepoint: Energies and properties calculated on a single molecular geometry.
    • Optimization: Molecular geometry optimization.
    • Gridoptimization: Geometry optimizations performed across a set of constrained distances and angles.
    • Torsiondrive: Geometry optimizations performed across a set of constrained torsion angles.
    • Manybody: Many-body computations, such as interaction energies.
    • Reaction: Computation of reaction energies.
    • NEB: Nudged Elastic Band computations for reaction profiles and transition states.

  9. Structure of NEB Datasets and Entries

    main

    NEB datasets are collections of NEB records. Each dataset consists of a NEBDatasetSpecification (which defines the default programs, methods, and basis sets) and multiple NEBDatasetEntry objects (which represent individual reaction chains).

    An NEBDatasetEntry contains:

    • name: The name of the entry.
    • initial_chain: An ordered list of molecules forming the chain.
    • additional_keywords: Per-entry keywords that override the dataset specification's NEBKeywords.
    • additional_singlepoint_keywords: Per-entry keywords that override the dataset specification's singlepoint_specification.keywords.
    • attributes: A user-defined metadata dictionary.
    • comment: A user-supplied comment.

  10. Project Permissions and Roles

    main

    Access to projects is governed by roles:

    • read role: Required to read projects.
    • submit role (or higher): Required to create, modify, and delete projects.

    Note on Scoping: Permissions are not scoped by ownership. A user with the submit role can modify or delete any project on the server, not just the ones they created.

  11. Use the local executor for non-HPC environments

    main

    If you are running on a standalone host, within a container (e.g., Kubernetes), or if your HPC cluster forbids long-running processes on the head node, use the local executor type.

    In this mode, you create a submission script that requests resources via your scheduler, activates the qcfractalcompute environment, and runs the qcfractal-compute-manager command itself. This allows the manager to run as a job rather than a persistent process on a login node.

    executors:
  local_executor:
    type: local
    max_workers: 4
    cores_per_worker: 16
    memory_per_worker: 96
    compute_tags:
      - '*'
    environments:
      use_manager_environment: False
      conda:
        - <worker_conda_env_name>
    worker_init:
      - source <absolute_path>/worker_init.sh

  12. Understand hard delete behavior for shared records and datasets

    main

    When using delete_records, delete_datasets, or delete_dataset_records, the system performs a hard delete (not a reversible soft delete).

    Hard deletes are subject to database integrity constraints: a record cannot be removed if it is still referenced by another project, a dataset, or a parent record. Because of this, these flags are best effort and fail quietly:

    • If a record is shared with another project or dataset, it will not be deleted. It is simply unlinked from the current project and remains on the server.
    • The server does not return information about which items failed to delete during a delete_project call.

    To ensure a record is actually gone, verify it using client.get_records(record_id, missing_ok=True). If it returns None, the record was successfully removed.

    # Attempt deletion
client.delete_project(7, delete_records=True)

# Verify if record 118326390 was actually removed
client.get_records(118326390, missing_ok=True)
# Returns None if successfully deleted