MatX Documentation

An Operator is the fundamental abstraction in MatX. Any type that implements the Operator interface can be used in arithmetic expressions, transforms, and assignments. This interface allows MatX to treat tensors, generators, and mathematical expressions uniformly.

To be considered an Operator, a type must implement the following functions:

• Size
• Shape
• Stride
• Rank
• operator() (required for rvalues; optionally for lvalues)

Because Operators are a superset, types like Tensors and Generators are all considered Operators and can be used anywhere an operator is accepted.

// Create a tensor. "t" is an operator
auto t    = make_tensor<float>({10});

// Create a sin operator that operates on "t" and name it "op"
auto op   = sin(t);

// Create a Hamming window generator and assign it to the variable "win"
auto win  = hamming({10});

// Launch a kernel where "win" is copied into "t"
(t = win).run();

For transforms that require expensive state (like a plan or a handle), use a cache to improve performance. This avoids re-creating the state for every call with the same signature.

To implement caching:

1. Define a Params Key: A struct containing all parameters that define the transform's state (e.g., dimensions, strides, stream, execution type).
2. Implement a Hash function (FftCUDAParamsKeyHash): Provides a quick hash for initial map lookups.
3. Implement an Equality function (FftCUDAParamsKeyEq): Performs a full comparison of all parameters once a hash match is found to ensure a true cache hit.
4. Use detail::GetCache().LookupAndExec<CacheType>(...) in your impl function. This method takes the cache ID, the parameters, a factory function to create the cached object on a miss, and an execution function to run the transform on a hit.

Example of the LookupAndExec pattern:

  using cache_val_type = detail::matxCUDAFFTPlan1D_t<decltype(out), decltype(in)>;
  detail::GetCache().LookupAndExec<detail::fft_cuda_cache_t>(
    detail::GetCacheIdFromType<detail::fft_cuda_cache_t>(),
    params,
    [&]() {
      return std::make_shared<cache_val_type>(out, in, stream);
    },
    [&](std::shared_ptr<cache_val_type> ctype) {
      ctype->Forward(out, in, stream, norm);
    }
  );

The proprietary_binary field allows downloading pre-built proprietary versions of a library. This is currently only supported for the nvcomp package.

How it works

The search logic follows this priority:

1. Search for a local version matching the version key (unless always_download is true).
2. Download the proprietary version if a valid <arch>-<os> key exists in the proprietary_binary dictionary and USE_PROPRIETARY_BLOB is set to ON.
3. Fallback to using the git_url and git_tag.

Configuration

The keys in the proprietary_binary dictionary must match the lowercase value of <arch>-<os>, where arch is CMAKE_SYSTEM_PROCESSOR and os is CMAKE_SYSTEM_NAME.

Placeholders

Supported placeholders in binary URLs include:

• ${rapids-cmake-version}
• ${cuda-toolkit-version}
• ${cuda-toolkit-version-major}
• ${cuda-toolkit-version-mapping}: Uses values from the proprietary_binary_cuda_version_mapping dictionary.
• $ENV{variable}

MatX uses lazy evaluation to perform element-wise operator fusion. Instead of executing arithmetic operations immediately and storing intermediate results in memory (which causes performance penalties due to high memory latency), MatX overloads operators to return objects that represent the computation.

When you define an expression, MatX builds a single C++ type representing the entire equation. The actual computation only occurs when you request a specific element or call .run(). This allows MatX to fuse multiple operations into a single pass, significantly reducing the number of loads and stores.

Key benefits:

• Reduced Memory Overhead: Avoids intermediate memory writes/reads.
• Readability: You can store complex expressions in variables without triggering execution.
• Transform Fusion: Complex transforms (like fft) can be fused with standard operators at compile-time.