Ginkgo Numerical Linear Algebra Library

Ginkgo is a production-ready sparse linear algebra library designed for high-performance computing (HPC) on GPU-centric architectures. It focuses on performance portability across various hardware, including:

• Multi-threaded CPUs
• NVIDIA GPUs
• AMD GPUs
• Intel CPUs/GPUs

The library is suitable for scientific applications requiring the solution of sparse linear systems through a variety of matrix formats, iterative (Krylov) solvers, and preconditioners.

Ginkgo's visual identity is based on a specific color palette and typography derived from its logo. When creating UI components, assets, or documentation that align with the Ginkgo brand, use the following hex codes and font specifications:

Color Palette

• ginkgo-orange-1 (#F7A71E): Suggested for use as a highlight color.
• ginkgo-orange-2 (#FCB725): Suggested for small elements like buttons.
• ginkgo-orange-3 (#FFCB05): Suggested for backgrounds.
• ginkgo-gray-1 (#6D6E71)
• ginkgo-gray-2 (#939598)
• ginkgo-gray-3 (#D1D3D4)

Typography

• The font used in the logo is Merriweather Black.

Ginkgo is organized into several functional modules that serve as the library's building blocks. When designing your application, you will interact with these primary abstractions:

• Executor: Defines the execution environment (where your code runs, e.g., CPU, GPU).
• LinOp (Linear Operations): Defines the mathematical operations to be performed. This includes:
  • Solvers: Algorithms to solve linear systems for a given matrix.
  • Preconditioners: Operations to precondition a system to improve solver performance.
  • Matrix Formats: Operations like sparse matrix-vector multiplication (SpMV) tailored to specific matrix storage formats.
• Log: Provides mechanisms to monitor code execution and performance.
• Stop: Manages the criteria used to determine when an iterative process should terminate.

Ginkgo is part of the xSDK effort and provides interoperability with major application libraries. Specifically, it provides wrappers for:

• deal.ii
• mfem

These wrappers allow users of these libraries to leverage Ginkgo's high-performance sparse linear algebra features directly within their existing workflows.

To perform a detailed analysis of solver performance, run the benchmark suite with the DETAILED=1 environment variable. This enables logging of the internal residual after every iteration and the time taken by all operations.

These features are implemented via loggers in ${ginkgo_src_dir}/benchmark/utils/loggers.hpp. Ginkgo provides hooks at critical code locations that can be inspected to track memory allocation sizes and other library aspects.

Ginkgo's design is built around two principal concepts to balance ease of use with high performance:

1. LinOp (Linear Operator) Interface: A class and object-oriented design where all solvers, preconditioners, and matrix formats are accessible through a common LinOp interface. This abstraction allows users to easily cascade solvers and preconditioners, tailoring them to specific needs regardless of the underlying device.
2. Low-level Device Kernels: Highly optimized, device-specific kernels (e.g., for NVIDIA or AMD GPUs) that use C++ templates to generate high-performance code for various parameters. These are separated from the high-level algorithms to ensure extensibility.

By using the LinOp interface, users can treat different mathematical objects uniformly across different hardware executors.

If you need to debug a specific test or see more detailed results for a failure, identify the test executable path from the make test output and run that binary directly from the build folder.

./path/to/test

The easiest way to run the complete suite is using the make benchmark command in your build directory. This invokes the benchmark/run_all_benchmarks.sh script.

You can configure the suite using environment variables. You can export them or pass them directly to make:

# Using export
export SYSTEM_NAME="V100"
make benchmark

# Passing on the same line
make benchmark SYSTEM_NAME="V100" EXECUTOR="cuda"

Ginkgo supports a HIP backend for AMD and NVIDIA GPUs. The build system attempts to auto-detect HIP at /opt/rocm/hip or via the HIP_PATH environment variable/CMake parameter.

To use nvcc instead of the default ROCm clang++ for HIP, set the HIPCXX environment variable:

export HIPCXX=nvcc

HIP Path Configuration

You can manually specify paths for HIP components using CMake variables or environment variables:

• ROCM_PATH: Sets the ROCM installation path (Default: /opt/rocm/).
• HIP_PATH: Sets the HIP installation path (Default: ${ROCM_PATH}/hip).
• HIP_CLANG_PATH: Sets the HIP-compatible clang binary path (Default: ${ROCM_PATH}/llvm/bin).
• HIPBLAS_PATH, HIPSPARSE_PATH, HIPFFT_PATH, ROCRAND_PATH, HIPRAND_PATH: Paths for specific HIP libraries.

Ginkgo supports several hardware acceleration modules. Enabling them requires additional dependencies:

CUDA Module

• CMake: 3.18+ (requires 3.22+ if installed via NVIDIA HPC Toolkit)
• CUDA: 11.0+ or NVHPC Package 22.7+
• Note: Host compiler restrictions imposed by your CUDA version apply.

HIP Module

• ROCm: 6.2.0+
• Packages: HIP, hipBLAS, hipSPARSE, hip/rocRAND, and rocThrust (all compiled with ROCm backend).
• CMake: 3.21+
• If hipFFT is available, it is used for FFT LinOps.

DPC++ (SYCL) Module

• oneAPI: 2024.1+
• Compiler: Set dpcpp or icpx as the CMAKE_CXX_COMPILER.
• Required oneAPI packages: oneMKL, oneDPL.

MPI Module

• MPI: 3.1+ (ideally GPU-Aware for optimal performance).

OpenMP Module (Windows)

• Requires MinGW.

Ginkgo uses a standard CMake build procedure. You can create a build directory, configure the project with desired options, and then build it.

For Microsoft Visual Studio users, you must specify the build configuration (e.g., Debug, Release, RelWithDebInfo, or MinSizeRel) using the --config flag during the build step.

mkdir build; cd build
cmake [OPTIONS] .. && cmake --build .