ZLUDA

The trace* samples use the following mechanism:

• They log output through the syelogd.exe daemon.
• They hook CreateProcessW to automatically load themselves into any child processes.

You can use withdll to inject a DLL into a process. For example, using withdll -d:traceapi.dll cmd.exe will launch a command shell where all subsequent processes log their API calls through traceapi.dll.

When zluda_trace runs, it creates a directory named after the command (e.g., add/). Inside this directory, you will find several files:

• log.txt: Contains a record of every CUDA library call, including arguments passed and the returned status code. It also captures calls to NVIDIA's "Dark API" (functions formatted as {INTERFACE_NAME}::function_name).
• module_NNNN_NN.elf: Precompiled SASS assembly for a single GPU architecture.
• module_NNNN_NN.ptx: Portable PTX assembly.
• module_NNNN_NN.log: (Optional) A compiler error log produced if ZLUDA's PTX compiler encounters an error while processing a PTX module. This is useful for identifying unsupported PTX instructions.

If you are filing a bug report, you should provide an archive of your trace logs.

Linux

Create a tar archive of the contents of your <LOG_DIRECTORY>:

tar -cvf logs.tar.gz -C <LOG_DIRECTORY> .

Windows

1. Navigate to %TEMP%\zluda in Windows Explorer.
2. Right-click the directory created for your application.
3. Select "Send to" -> "Compressed (zipped) folder" (exact steps may vary by Windows version).

To achieve native performance when using ZLUDA with llama.cpp, you should compile it for CUDA architecture 86 with cuBLAS enabled. While you can compile for multiple architectures, ensure that at least one of them is 80, 86, or 89. Disabling cuBLAS may result in performance degradation.

cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="86" -DGGML_CUDA_FORCE_CUBLAS=true

You can use the vcpkg dependency manager to install Detours. Follow these steps to bootstrap vcpkg and install the package:

1. Clone the vcpkg repository.
2. Bootstrap vcpkg.
3. Integrate vcpkg with your build environment.
4. Install the detours port.

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
vcpkg install detours

To avoid building HiGHS from source, you can link against an existing installation on your system.

1. Install pkg-config on your system.
2. Enable the discover feature in your Cargo.toml.

Note: This method typically results in dynamic linking, meaning HiGHS must be present on any system where you deploy your application. Since HiGHS is not widely available in many package managers, you may still need to build and install it from source manually before using this feature.

When running large applications, you can use zluda_precompile to avoid long initial launch delays. This tool scans a specified directory or file, extracts all GPU code, compiles it, and saves the results to the cache. This ensures the GPU code is already cached when the application is launched for the first time.

Key characteristics:

• Performance: It uses all available machine threads to speed up the compilation process compared to the application's default behavior.
• Trade-off: It may compile more code than the application actually requires, which might make the precompilation process itself slower than if the application handled it incrementally.

# Windows
zluda_precompile.exe <PATH>

# Linux
zluda_precompile <PATH>

You can test the samples using nmake:

• Individual samples: Navigate to a specific sample directory and run nmake test.
• All samples: Navigate to the main samples directory and run nmake test.

Note: Some samples are architecture-specific. Tests for these will be skipped if run on an unsupported architecture. Most executables also support the /? flag to display usage information.

# Run all tests from the samples directory
cd path/to/detours/samples
nmake test

# View usage for a specific executable
executable_name.exe /?

By default, highs-sys builds HiGHS and links it statically. To do this, you must have a C++ compiler and cmake installed.

Linux (Debian)

sudo apt install g++ cmake

macOS

1. Install C++ compiler: xcode-select --install
2. Install cmake: brew install cmake
3. (Optional) If using libz or ninja features, install them via brew: brew install ninja (and ensure libz is available).

Windows

Install cmake and Clang (via LLVM) using winget:

winget install -e --id Kitware.CMake
winget install -e --id LLVM.LLVM

If the ninja feature is enabled:

winget install -e --id Ninja-build.Ninja

Note on libz: If you enable the libz feature, you must either add libz-sys as a dependency in your project or manually set the ZLIB_ROOT environment variable.