Kokkos Kernels Documentation

repository·develop·Indexed 18 days ago

https://github.com/kokkos/kokkos-kernels

High-performance, performance-portable computational kernels for linear algebra (BLAS, Sparse BLAS) and graph operations using the Kokkos programming model. Features include the GMRES solver for sparse linear systems, Distance 2 Graph Coloring (D2GC) routines, and a framework for BLAS performance testing. Designed as a local, coarse-grained building block for shared-memory parallel programming.

Tokens
104.2K
Snippets
192
Records
339
Agent score
60%

What's inside Kokkos Kernels

  1. Overview of Kokkos Kernels Graph Package

    develop

    The graph package provides high-performance implementations of graph coloring algorithms. These algorithms are used as a foundation for a symmetric multithreaded Gauss-Seidel implementation.

    Key capabilities include:

    • Graph Coloring Algorithms: Core algorithms for coloring graphs.
    • Symmetric Multithreaded Gauss-Seidel: An implementation that leverages graph coloring to enable efficient parallel execution.
  2. Overview of Kokkos Kernels

    develop

    Kokkos Kernels provides local computational kernels for linear algebra and graph operations using the Kokkos shared-memory parallel programming model.

    Key characteristics:

    • Local: It does not use MPI directly; it runs within a single MPI process or stand-alone. It can serve as a building block for MPI-aware libraries like Tpetra.
    • Coarse-grained: The kernels are designed to perform significant work that justifies parallelization via Kokkos.
    • Performance Portability: It implements BLAS, Sparse BLAS, and Graph Kernels.

    Available Kernels include:

    • (Multi)vector dot products, norms, and AXPY-like updates.
    • Sparse matrix-vector multiply (SpMV) and other sparse/dense kernels.
    • Sparse matrix-matrix multiply (SpGEMM).
    • Graph coloring and Gauss-Seidel with coloring.

    Important Usage Note: Do NOT use or rely on the KokkosBlas::Impl namespace or anything in the src/impl/ directory. Use only the public interfaces located in src/.

  3. Overview of Sparse API in Kokkos Kernels

    develop

    The Kokkos Kernels Sparse API provides data structures and high-performance kernels for sparse linear algebra. Sparse containers store only non-zero indices and values, assuming all other entries are zero to save memory. The API is organized into several functional categories:

    • Containers: Specialized data structures for storing sparse data (e.g., CRS, BSR, CCS, COO).
    • Sorting: Utilities to sort and merge sparse data structures and graphs.
    • Linear Algebra Operations: Kernels for SpMV (Sparse Matrix-Vector Multiplication), SpAdd (Matrix-Matrix Addition), SpGEMM (Matrix-Matrix Multiplication), and SpTRSV (Triangular Solve).
    • Linear Solvers / Preconditioners: Implementations of Gauss-Seidel, SOR, and Incomplete LU (ILU) factorizations.
    • Utility Functions: Specialized tools like RCB (Recursive Coordinate Bisection) block extraction.
  4. Use the KokkosSparse Preconditioner Interface

    develop

    The KokkosSparse_Preconditioner class in the KokkosSparse::Experimental namespace provides an abstract base class for using Kokkos-based preconditioners with iterative linear solvers (such as the GMRES implementation in examples/gmres). It is designed to be compatible with other solver packages and is loosely based on the Trilinos IfPack2::Preconditioner class.

    Template Parameters

    • CRS: The type of the compressed row sparse matrix. All key types should be derivable from this type.
  5. Use KokkosGraph Load-Balance Routines

    develop

    KokkosGraph load-balance routines map irregular task sizes onto a dense work-item space. These routines are useful for redistributing work from tasks of varying sizes into a uniform space of work items.

    Core Abstractions

    LoadBalanceResult<View> This structure stores the mapping for each work item:

    • tasks: A view containing the source task index for each work item.
    • ranks: A view containing the rank (index) of that work item within its source task.

    When to use which routine

    • load_balance: Use this when you are starting from raw task sizes (a view of work amounts per task).
    • load_balance_exclusive: Use this when you already have an exclusive prefix sum (scan) of the task sizes available.
    • load_balance_team: Use this for team-local cooperative load balancing within a Kokkos team.
    • inclusive_prefix_sum_team: Performs an inclusive prefix sum locally within a team.
    #include <Kokkos_Core.hpp>
    #include <KokkosGraph_LoadBalance.hpp>
    
    using view_type = Kokkos::View<int*>;
    
    void example(const view_type& taskSizes) {
      auto result = KokkosGraph::load_balance(taskSizes);
      (void)result;
    }
  6. Use KokkosGraph Merge-Path Routines

    develop

    KokkosGraph provides routines to traverse the merge path induced by two ordered rank-1 inputs. Instead of generating a materialized merged output, these routines invoke a user-provided stepper for every step along the path. This is useful for performing custom work during a merge operation without the overhead of allocating a new merged view.

    Available Routines

    1. merge_path_thread: Used when the caller wants to perform custom per-step work within a single thread. It allows forwarding optional contexts (ctxs) to the stepper.
    2. merge_path_team: Partitions the merge path across a Kokkos team. It forwards a thread-level context to each stepper invocation.

    Core Abstractions

    • StepperContext: A struct recording the current positions in the inputs and the path:
      • ai: Position in the first input (a).
      • bi: Position in the second input (b).
      • pi: Overall position in the path.
    • StepDirection: An enum indicating which input is being consumed in the current step:
      • StepDirection::a: The step consumes an element from input a.
      • StepDirection::b: The step consumes an element from input b.
    // Example of a basic thread-level merge path traversal
    #include <Kokkos_Core.hpp>
    #include <KokkosGraph_MergePath.hpp>
    
    template <class AView, class BView>
    void example(const AView& a, const BView& b) {
      auto stepper = [](KokkosGraph::StepDirection dir, 
                        KokkosGraph::StepperContext step) {
        (void)dir;
        (void)step;
      };
      KokkosGraph::merge_path_thread(a, b, a.size() + b.size(), stepper);
    }
  7. Type requirements for symmetric_gauss_seidel_apply

    develop

    When using symmetric_gauss_seidel_apply, ensure your types satisfy the following constraints:

    Consistency with KernelHandle

    The input parameter types must match the types defined by the KernelHandle:

    • row_map value type must match KernelHandle::const_size_type.
    • entries value type must match KernelHandle::const_nnz_lno_t.
    • values value type must match KernelHandle::const_nnz_scalar_t.
    • y_rhs_input_vec value type must match KernelHandle::const_nnz_scalar_t.
    • x_lhs_output_vec value type must match KernelHandle::nnz_scalar_t.

    Layout Requirements

    The views describing the matrix (row_map, entries, and values) must not use Kokkos::LayoutStride. They should use a non-strided layout (e.g., Kokkos::LayoutLeft or Kokkos::LayoutRight).

  8. Understand Kokkos Kernels CMake option naming convention

    develop

    When developing or configuring Kokkos Kernels, be aware of the distinction between user-visible cache variables and internal regular variables:

    1. User-visible Cache Variables: Prefixed with KokkosKernels_ (e.g., KokkosKernels_ENABLE_THING). These are intended to be set by the user via the command line or ccmake.
    2. Internal Regular Variables: All-caps versions without the prefix (e.g., KOKKOSKERNELS_ENABLE_THING). These are used within the CMake logic to implement the actual configuration and are not intended for direct user manipulation in the cache.
  9. Manage sub-handles in KokkosKernelsHandle

    develop

    The KokkosKernels::Experimental::KokkosKernelsHandle provides a generic set of get, create, and destroy member functions to manage specialized sub-handles associated with specific algorithms. These sub-handles allow for fine-grained control over the state and configuration required by individual kernels (like SpGEMM, SpADD, or GMRES) without cluttering the main handle.

    // General pattern for sub-handle management
    KernelHandeType* get_kernel_handle();
    void create_kernel_handle(Args...);
    void destroy_kernel_handle();
  10. Understand the relationship between Component Compilation and Test Registration

    develop

    KokkosKernels uses two independent mechanisms for components:

    1. Compilation Control (ENABLE_COMPONENT_* and ENABLE_ALL_COMPONENTS): These BOOL flags determine which library source files are actually compiled into the library.
    2. Test Registration Control (ENABLED_COMPONENTS): This STRING flag determines which component tests are registered with the test runner.

    Key Interaction Rules:

    • ENABLED_COMPONENTS has no effect on which source files are compiled.
    • ENABLE_COMPONENT_* flags do not affect which tests are registered.
  11. Requirements for KokkosGraph Merge Routines

    develop

    To use the KokkosGraph merge routines, the following conditions must be met:

    Type Requirements

    • AView, BView, and CView must be rank-1 Kokkos views.
    • The input value types must support the ordering relation used by the merge (e.g., non-decreasing order).

    Memory and Size Requirements

    • Input Order: Inputs must be assumed to be sorted in non-decreasing order.
    • Output Size: For merge_into overloads, c.size() must exactly equal a.size() + b.size().
    • Accessibility: The memory spaces of the inputs and output must be accessible from the selected execution space.

    Edge Cases

    • If one input is empty, the result is a copy of the other input.
  12. Understand Graph coloring categories in Kokkos Kernels

    develop

    Kokkos Kernels provides generic graph coloring capabilities used for applications like Gauss-Seidel and multigrid aggregation. The algorithms are categorized into two main types based on the coloring constraint:

    1. Distance 1 coloring: Ensures each node has a different color than all of its immediate neighbors.
    2. Distance 2 coloring: Ensures each node has a different color than its neighbors AND its neighbors' neighbors.

    Additionally, the library provides a Graph coloring handle which extends the standard KokkosKernels handle to include options specific to these coloring algorithms.