Kokkos Core

The Kokkos build system is based on Modern CMake. This documentation is intended for developers who need to modify the build system, such as adding new CMake options that influence header configuration macros, optional features, third-party libraries, or compiler and linker flags.

Note for end-users: If you are looking for general usage, installation, or API documentation, please refer to the official documentation at https://kokkos.org.

The Kokkos Tuning System is designed to move away from hardcoded tuning parameters (like CUDA block sizes) toward a callback-based system. This allows external tools to intelligently tune parameters for specific applications and architectures by providing context and receiving feedback.

Key goals of the system:

• Expose Context: Provide tools with application features (e.g., kernel name, execution space, or application-specific variables like matrix_size) to enable intelligent tuning.
• Expose Tuning Parameters: Describe tuning parameters with metadata including Type (int, float, string), Semantics (categorical, ordinal, interval, or ratio), and Candidates (sets or ranges).
• Feedback Loop: Allow tools to make decisions, measure performance, and learn from results.
• Zero Overhead: Ensure no perturbation of Kokkos Core when the tuning system is disabled.

Kokkos configuration is managed through CMake cache variables. All options follow the pattern Kokkos_ENABLE_<OPTION_NAME>. You can set these during the configuration step using the -D flag.

Example: To enable tests, use -DKokkos_ENABLE_TESTS=ON.

When Kokkos_ENABLE_TESTS is ON, Kokkos generates smoke test targets for each enabled backend, such as Kokkos_CoreUnitTest_<Backend>_SmokeTest.

You can obtain Kokkos via curl, wget, or git.

To download a specific release (e.g., version 5.1.0):

curl -OJ -L https://github.com/kokkos/kokkos/releases/download/5.1.0/kokkos-5.1.0.tar.gz
# Or with wget
wget https://github.com/kokkos/kokkos/releases/download/5.1.0/kokkos-5.1.0.tar.gz
# Or with git
git clone --depth=2 --branch 5.1.0 https://github.com/kokkos/kokkos.git

To clone the latest development version:

git clone --branch develop  https://github.com/kokkos/kokkos.git

If you use the Spack package manager, you can install Kokkos using the following command. You can view available configuration options for the package using spack info kokkos.

spack install kokkos

To use Kokkos in your own project, use find_package(Kokkos) to locate the package and then link against the Kokkos::kokkos target using target_link_libraries. Because Kokkos uses modern CMake, all necessary include paths, C++ standard flags, and architecture-specific optimizations are automatically propagated to your target.

find_package(Kokkos)
add_library(stencil stencil.cpp)
target_link_libraries(stencil Kokkos::kokkos)

To make tuning tools aware of your application's internal state (e.g., number of active particles), use the Kokkos::Tools::Experimental namespace.

Workflow:

1. Declare Variables: Use declare_input_type to tell tools about variables you will provide. This returns a size_t ID.
2. Manage Contexts: Use get_new_context_id() or get_current_context_id() to obtain a context ID.
3. Start Context: Call begin_context(context_id) to signal the start of a region (tools may start timers here).
4. Set Values: Use set_input_values to pass values to the tool using IDs obtained from make_variable_value.
5. End Context: Call end_context(context_id) to signal the end of the region.

To add custom preprocessor macros (e.g., #define Kokkos_X) to the Kokkos configuration header, follow these steps:

1. Add #cmakedefine Kokkos_X to the KokkosCore_config.h.in file.
2. Define the variable in CMake using KOKKOS_OPTION(X ON "Description") or by setting it via SET(Kokkos_X ON) in a CMakeLists.txt.
3. If setting the variable within a function, ensure it is visible to the top-level scope using PARENT_SCOPE.

The tuning system uses contextId to define the scope and lifetime of context variables and tuning decisions. When a context begins, declarations are valid; when it ends, the tool is expected to take measurements and learn from the tuning values provided during that window.

To manage the lifecycle of a context, use startContext and endContext with a specific contextId.

Example Workflow:

startContext(contextId(0))
declare_value("is_safe_to_push_button", true, contextId(0));
foo();
endContext(contextId(0));
bar();

In the example above, the variable is_safe_to_push_button is only valid within the scope of contextId(0). Once endContext(contextId(0)) is called, the state is no longer valid in bar().

startContext(contextId(0))
declare_value("is_safe_to_push_button",true,contextId(0));
foo();
endContext(contextId(0));
bar();