DOCSEARCHbyAlgolia
Submit Docs

Bamboost Documentation

website·Indexed 30 days ago

https://bamboost.ch/

A framework for managing simulation data, featuring tools for HDF5 file mapping, simulation indexing, and collection management. It includes a CLI for indexing and configuration, MPI utilities, and specialized plugins such as FEniCS support. The system handles simulation metadata, field and global data, and provides mechanisms for reading, writing, and setting up simulation runs.

Tokens
34.5K
Snippets
233
Records
293
Agent score
50%

What's inside Bamboost

  1. Overview of bamboost simulation data framework

    bamboost is a Python framework designed for managing scientific simulation data. It organizes data into collections, which contain individual simulations. Each simulation is stored as a directory containing a central data.h5 HDF5 file for parameters and results, allowing for additional files (e.g., CSVs, images, or 3rd party software output) to be stored alongside the data.

    Key principles include:

    • Independence: Datasets are self-contained and understandable without external dependencies.
    • Path Redundancy: Data is referenced via unique identifiers (UIDs) rather than absolute paths, ensuring that moving files or renaming directories does not break data references.
    • Indexing: While the primary storage is the filesystem, bamboost uses a SQLite database as a cache to allow fast querying of simulations and parameters. This cache can be automatically rebuilt if deleted or corrupted.

  2. Overview of the bamboost plugin system

    bamboost provides a lightweight plugin system that allows third-party packages to replace core components with specialized subclasses without monkey-patching. This is primarily used by package authors to ship solver-specific writers (e.g., FEniCS writers) that integrate with the bamboost data model. The main targets for replacement are Collection and Simulation / SimulationWriter.

  3. Overview of BAMBOOST Indexing

    BAMBOOST uses a SQLite database to index collections and their associated simulations and parameters. The index stores the paths of collections (identified by unique UIDs), metadata, and simulation parameters. The system is designed for parallel execution: while it works in parallel, database operations are executed only on the root process, and results are broadcast (bcast) to all other processes.

  4. Use the FenicsBamboostPlugin for HDF5 data export

    The FenicsBamboostPlugin is used to write FEniCS data to HDF5 files. It provides access to the following internal components:

    • FenicsWriter: Handles the step-writing process (_FenicsStepWriter).
    • FenicsFieldData: Manages field data (_FenicsFieldData).
    • FenicsMeshes: Manages mesh data (_FenicsMeshes).

  5. Manage simulation metadata in bamboost

    Every simulation in bamboost contains a metadata dictionary stored as attributes on the root group (/) of the data.h5 file. This dictionary stores information about the simulation rather than the results. While any key is valid, bamboost automatically handles a set of reserved keys: status (lifecycle state), created_at (creation timestamp), description (free-text), submitted (boolean indicating if the run script was submitted), and tags (list of strings).

  6. Manage simulation data using Series

    In bamboost, a Series is used to store simulation data, typically as a timeseries of steps. Series are coupled to meshes and are stored as groups within an HDF file. By default, every simulation has a root-level series available at sim.data. Multiple custom series can be created if needed.
    # Access the default series
print(sim.data)

# Create or access a custom named series
my_series = sim.require_series("my_series")

  7. Index BAMBOOST collections and simulations

    The bamboost.index module allows developers to index BAMBOOST collections and simulations. It uses an SQLite database via SQLAlchemy to cache information about collections and simulations, enabling efficient retrieval of simulation paths and metadata.
    from bamboost.index import Index

# Initialize the index
index = Index()

# Scan for collections in known paths
index.scan_for_collections()

# Resolve the path of a collection using its unique identifier (UID)
path = index.resolve_path('<collection-uid>')

# Retrieve a specific simulation using the collection UID and simulation name
simulation = index.get_simulation('<collection-uid>', '<simulation-name>')

  8. Understand HDF Object hierarchy in bamboost

    In bamboost, HDF files are hierarchical. They consist of groups (which can contain other groups or datasets) and datasets. Both are referred to as objects. Every object can have attributes used for metadata. In the bamboost implementation, these objects are references defined by a file and an in-file path, all deriving from the H5Reference base class.

  9. Understand HDF file handling in bamboost

    bamboost provides a public interface for handling HDF files as an abstraction of h5py. Its primary goals are to simplify file handling (automated opening and closing), provide a Python-native in-memory-like experience for data R/W, and ensure an MPI-compatible interface.

  10. Understand the bamboost simulation data model

    In bamboost, every collection consists of multiple simulations. Each simulation is centered around a primary HDF5 file named data.h5, which stores the simulation's metadata, parameters, and numeric data. A simulation may also include an executable for computing results, a bash script for execution or HPC scheduling, and other user-relevant files.

  11. Use bamboost.mpi.serial for non-MPI environments

    The bamboost.mpi.serial module acts as a mock for mpi4py.MPI. It is used as the default communicator when config.options.mpi is set to False or when the application is not running under an MPI launcher. Using this mock instead of importing real MPI significantly increases launch speed for CLI applications.

  12. bamboost.cli.app module overview

    The bamboost.cli.app module implements the command-line interface for bamboost using the typer library. It provides functionality to list and create collections and simulations, run simulations, and handle database migrations.