repository·main·Indexed 42 days ago
https://github.com/rapidsai/kvikioA high-performance file I/O library providing Python, C++, and Java bindings to NVIDIA's cuFile API. It enables GPUDirect Storage (GDS) for high-speed data transfers between storage and GPU memory, featuring a Python Zarr backend, non-blocking API support, and transparent handling of host and device memory with a seamless fallback to POSIX I/O.
The Java KvikIO bindings provide access to GDS (GPUDirect Storage) via the underlying cuFile API.
Current Capabilities:
Limitations:
KvikIO is a high-performance file I/O library providing Python and C++ bindings to NVIDIA's cuFile. It enables GPUDirect Storage (GDS) for high-speed data transfers between storage and GPU memory.
Key features include:
NVIDIA KvikIO is a high-performance file I/O library providing Python and C++ bindings to cuFile. It is designed to enable GPUDirect Storage (GDS) for high-speed data transfers between storage and GPU memory.
Key characteristics:
cuFile for GPUDirect Storage when available.KvikIO supports direct access to remote files using various protocols, including:
You can perform I/O operations directly on these remote resources as if they were local files.
KvikIO supports two backends for remote (HTTP/S3/WebHDFS) reads, configured via KVIKIO_REMOTE_IO_BACKEND:
EASY_THREADPOOL (default): Uses the Libcurl easy API within the KvikIO thread pool. Concurrency is limited by the number of threads in the pool (KVIKIO_NTHREADS). Each thread blocks on curl_easy_perform() for a single transfer.
MULTI_POLL: Uses the Libcurl multi API driven by $N$ reactor threads. A single reactor multiplexes many in-flight handles. Concurrency is bounded by KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS rather than the thread pool size.
Use kvikio.defaults.set('remote_io_backend', ...) to switch. Note that kvikio.RemoteFile.pread reads this setting on every call, so changes apply to subsequent reads.
import kvikio
# Switch to the high-concurrency MULTI_POLL backend
kvikio.defaults.set('remote_io_backend', 'MULTI_POLL')KvikIO distinguishes between wall time and busy time to provide more accurate performance metrics:
busy time rather than total wall time. This prevents I/O speed from appearing artificially low due to computation gaps.To recover the whole-span rate (the rate relative to wall time), multiply the busy bandwidth by the Summary.busy_fraction attribute.
When creating a kvikio.SummaryMonitor, you can choose the ObservationKind to change how operations and durations are counted:
kvikio.ObservationKind.LOGICAL (Default): Counts one operation per user-facing call. If KvikIO splits a single call across a thread pool, it is treated as one row. The span runs from the moment the call is submitted until it completes. This is best for understanding how many I/Os your program issued and how long each took from the program's perspective.kvikio.ObservationKind.PHYSICAL: Counts one operation per actual transfer (e.g., one thread-pool task or one HTTP range request). The span starts when a worker picks up the task or the request goes on the wire. This is best for measuring device/link utilization and actual bandwidth over time.Note: Both kinds report the same total bytes and errors, but they differ in operation counts and durations. busy time (the union of spans) remains the same, but total_duration (the sum of spans) will differ because LOGICAL includes queueing time.
# Create a monitor for logical user-facing calls
calls = kvikio.SummaryMonitor(kvikio.ObservationKind.LOGICAL)
# Create a monitor for physical underlying transfers
transfers = kvikio.SummaryMonitor(kvikio.ObservationKind.PHYSICAL)KvikIO's compatibility mode determines whether the library uses libcufile.so (for GPUDirect Storage) or standard POSIX I/O. This is controlled by the KVIKIO_COMPAT_MODE environment variable or kvikio::defaults::set_compat_mode().
ON (TRUE, YES, 1): Enables compatibility mode (uses POSIX).OFF (FALSE, NO, 0): Disables compatibility mode and enforces cuFile I/O. If the system does not support cuFile, operations may error out.AUTO: Attempts cuFile I/O first, falling back to POSIX if requirements are not met (e.g., libcufile.so is missing, running in WSL, or /run/udev is unreadable).Note: This is distinct from cuFile's own internal compatibility mode.
The kvikio_nic plugin is included by default in both the libkvikio conda package and the libkvikio wheel. It consists of an executable (kvikio_nic_nsys_plugin) and a manifest (nsys-plugin.yaml) located in a kvikio_nic directory.
You can find the installation path using the kvikio.nsys_plugin_search_dir() function.
import kvikio
print(kvikio.nsys_plugin_search_dir())For Nsight Systems version 2026.2.1 and later, use the NSYS_PLUGIN_SEARCH_DIRS environment variable to point to the KvikIO plugin directory. This avoids the need to modify the Nsight Systems installation directory.
export NSYS_PLUGIN_SEARCH_DIRS="$(python -c 'import kvikio; print(kvikio.nsys_plugin_search_dir())')"
nsys profile --enable=help # should list kvikio_nic
nsys profile --enable=kvikio_nic,--device=eth0 --trace=cuda,nvtx,osrt -o nsys_report my_applicationTo use or compile the Java KvikIO bindings, ensure the following environment requirements are met:
jcuda-12.0.0.jarjcuda-natives-12.0.0.jarYou can compile the shared library and run the included tests/examples using Maven.
Important Note on File Paths:
By default, examples create temporary files in /tmp. If your /tmp directory is not mounted in a way compatible with GDS, the example will run in compatibility mode. To test true GDS performance, create a file on a compatible mount point (e.g., /mnt/nvme/) and update the file path in src/test/java/ai/rapids/kvikio/cufile/BasicReadWriteTest.java.
Important Note on Library Paths:
When running examples, ensure that the JCuda shared library files and the libCuFileJNI.so file are included in your JVM library path.
# 1. Prepare a compatible test file (replace /mnt/nvme/ with your mount)
touch /mnt/nvme/java_test
# 2. Compile the shared library and run tests
cd kvikio/java/
mvn clean install
# 3. To just rerun the tests
cd kvikio/java/
mvn test