DOCSEARCHbyAlgolia
Submit Docs

OpenCoarrays

repository·main·Indexed 42 days ago

https://github.com/sourceryinstitute/opencoarrays

A parallel ABI for gfortran that enables Coarray Fortran support by abstracting communication libraries like MPI. It consists of the caf compiler wrapper, the libcaf_mpi runtime library, and the cafrun program launcher, allowing developers to write portable parallel Fortran code.

Tokens
8.9K
Snippets
31
Records
42
Agent score
48%

What's inside OpenCoarrays

  1. Overview of OpenCoarrays

    main

    OpenCoarrays provides a parallel application binary interface (ABI) for the GNU Compiler Collection (GCC) Fortran compiler (gfortran). It enables support for Coarray Fortran (parallel programming features) by abstracting away the underlying communication library (typically MPI). This allows Fortran programs to use parallel features without being hard-coded to a specific communication backend.

    OpenCoarrays consists of three main components:

    1. caf: A compiler wrapper.
    2. libcaf_mpi: A parallel runtime library.
    3. cafrun: A program launcher.

    These components provide a uniform abstraction for compiling and executing Coarray Fortran code without requiring direct reference to the MPI layer.

  2. Understand the OpenCoarrays ABI and Runtime Libraries

    main

    OpenCoarrays provides a transparent Application Binary Interface (ABI) that allows applications to use interchangeable parallel runtime libraries. This abstraction enables the same application code to run on different communication backends.

    Supported Communication Libraries

    The runtime libraries are based on:

    • MPI (Message Passing Interface) — This is the only currently maintained, built, and tested runtime.
    • OpenSHMEM — Source code is provided for research documentation purposes only.
    • GASNet — Source code is provided for research documentation purposes only.

    Note: For a Fortran 2018 parallel runtime library based on GASNet-EX, use the Caffeine repository instead.

    ABI Header Files

    The ABI is defined by the following header files (listed in dependency order, from top to bottom):

    1. libcaf.h
    2. libcaf-gfortran-descriptor.h
    3. libcaf-version-def.h

  3. OpenCoarrays ABI Overview for Compiler Developers

    main

    The OpenCoarrays Application Binary Interface (ABI) defines how a compiler accesses coarray functionality.

    Note for Application Developers: Most application developers do not need to interact with this ABI directly. You should write standard-conforming Fortran 2008 or 2018 code and compile it using the OpenCoarrays caf compiler wrapper. The wrapper handles the ABI interactions automatically.

    Note for Compiler Developers: Function names in the source code may include a vendor-specific PREFIX to avoid name clashes.

  4. OpenCoarrays prerequisite dependency tree

    main

    When installing from source, the following dependency hierarchy must be satisfied:

    opencoarrays
├── cmake-3.4.0
└── mpich-3.2
    └── gcc-6.1.0
        ├── flex-2.6.0
        │   └── bison-3.0.4
        │       └── m4-1.4.17
        ├── gmp
        ├── mpc
        └── mpfr

  5. OpenCoarrays ABI Types and Definitions

    main

    The ABI defines several key types and status values used for coarray management:

    caf_token_t

    An opaque pointer used to identify a coarray. On the compiler side, it is a void *. The exact internal content is implementation-defined by the library.

    caf_register_t

    An enumeration indicating the type of coarray variable being registered:

    • CAF_REGTYPE_COARRAY_STATIC: Non-allocatable coarrays.
    • CAF_REGTYPE_COARRAY_ALLOC: Allocatable coarrays.
    • CAF_REGTYPE_LOCK_STATIC: Non-allocatable lock variables.
    • CAF_REGTYPE_LOCK_ALLOC: Allocatable lock variables.
    • CAF_REGTYPE_CRITICAL: Lock variables used for critical sections.
    • CAF_REGTYPE_EVENT_STATIC: Non-allocatable event variables.
    • CAF_REGTYPE_EVENT_ALLOC: Allocatable event variables.

    Stat Values

    Used to indicate the status of locks or images:

    • STAT_UNLOCKED: 0
    • STAT_LOCKED: 1
    • STAT_LOCKED_OTHER_IMAGE: 2
    • STAT_STOPPED_IMAGE: 6000
    typedef enum caf_register_t {
  CAF_REGTYPE_COARRAY_STATIC,
  CAF_REGTYPE_COARRAY_ALLOC,
  CAF_REGTYPE_LOCK_STATIC,
  CAF_REGTYPE_LOCK_ALLOC,
  CAF_REGTYPE_CRITICAL,
  CAF_REGTYPE_EVENT_STATIC,
  CAF_REGTYPE_EVENT_ALLOC
} caf_register_t;

#define STAT_UNLOCKED           0
#define STAT_LOCKED             1
#define STAT_LOCKED_OTHER_IMAGE 2
#define STAT_STOPPED_IMAGE      6000

  6. Access the underlying MPI communicator via the opencoarrays module

    main

    While the OpenCoarrays ABI is designed to be transparent and hide the underlying communication layer, the opencoarrays module provides a specific exception. It includes a public function that allows you to retrieve the underlying MPI communicator.

    This is useful when you want to use coarray Fortran to drive an ensemble of simulations that rely on pre-existing MPI infrastructure.

  7. Install OpenCoarrays using EasyBuild

    main

    Use EasyBuild to search for and automatically install OpenCoarrays using specification files (easyconfigs).

    Commands:

    1. Search for available easyconfigs: eb --search OpenCoarrays.
    2. Install using a specific easyconfig (e.g., OpenCoarrays-1.9.0-gompi-2017a.eb) with the --robot flag to automate prerequisites: eb <easyconfig_name> --robot.
    3. Load the environment: module load OpenCoarrays/<version_string>.
    eb --search OpenCoarrays
eb OpenCoarrays-1.9.0-gompi-2017a.eb --robot
module load OpenCoarrays/1.9.0-gompi-2017a

  8. Run OpenCoarrays with more images than CPU cores using OpenMPI

    main

    If you are using OpenMPI and need to run a Coarray program with more images (MPI ranks) than available logical CPU cores, OpenMPI requires the --oversubscribe flag. You can pass this flag directly to the cafrun wrapper, which will forward it to the underlying mpirun command.

    Example invocation:

    cafrun -np 32 --oversubscribe ./a.out <arg1> <arg2>
    cafrun -np 32 --oversubscribe ./a.out <arg1> <arg2>

  9. Install OpenCoarrays via Virtual Machine

    main
    Users on macOS, Windows, or Linux can use a pre-configured Lubuntu Linux virtual machine provided by the Sourcery Institute. This VM runs inside [VirtualBox] and comes pre-installed with [GCC], [MPICH], OpenCoarrays, and various other open-source packages for modern Fortran development.

  11. Install OpenCoarrays

    main

    For detailed installation instructions, please refer to the INSTALL.md file in the repository.

    Alternatively, you can try OpenCoarrays online using a Jupyter notebook kernel via Binder. This requires no local downloads, configuration, or installation. The default index.ipynb notebook is read-only, but you can execute it, copy it to make changes, or create a new CAF kernel notebook.

    https://bit.ly/CAF-Binder

  12. Build OpenCoarrays from source using Make

    main

    If CMake is unavailable, you can build the OpenCoarrays parallel runtime library using the static Makefiles in the src directory.

    Note: This method is less portable and does not generate the caf compiler wrapper, the cafrun launcher, or the opencoarrays module. You must manually move the resulting library to your installation path.

    To enable efficient strided-array transfer support, you must uncomment the -DSTRIDED flag in the make.inc file before building.

    tar xvzf opencoarrays.tar.gz
cd opencoarray/src
make
mv mpi/libcaf_mpi.a <insert-install-path>