GPU.js

When writing the orchestration function for createPipeline, you must follow strict trace-time rules. Violations will throw an error at build time (during the first call).

Prohibited operations during orchestration (trace-time):

• Reading handles: You cannot read an element or property of a handle (e.g., handle[0]). This throws: "pipeline intermediate results cannot be read during orchestration".
• Arithmetic/Conditions on handles: Using a handle in arithmetic or conditional logic (via valueOf or Symbol.toPrimitive) is forbidden.
• Non-deterministic functions: Calling Math.random() is forbidden; orchestration must be deterministic.
• External consumption: Calling any function that consumes a handle (other than a GPU.js kernel created by the same GPU instance) is forbidden.

Legal operations:

• Kernel calls: Calling GPU.js kernels created by the same GPU instance using handles or plain JS values as arguments.
• Non-handle arguments: Using numbers or arrays (uploaded per call) as kernel arguments.
• Captured values: Plain values captured via closures are frozen into the plan at trace time.

GPU.js kernels can accept several types of input arguments directly in the kernel function. Supported types include:

• Numbers: Scalar values.
• Arrays: 1d, 2d, or 3d arrays of numbers. Supported array types include Array, Float32Array, Int16Array, Int8Array, Uint16Array, and UInt8Array.
• HTML Elements: Single HTMLImageElement, an array of images (HTMLImageArray), or a HTMLVideoElement (V2+).
• Pre-flattened Arrays: For faster data upload, use the input function to pass a flattened array with specified dimensions.

Memory Layout for input(): When using input(flattenedArray, [width, height, depth]), the dimensions follow [x, y, z] where x is the fastest-varying (innermost) index. A kernel access arg[i][j][k] maps to z = i, y = j, x = k. This means input(flat, [X, Y, Z]) is equivalent to a nested array of shape [Z][Y][X].

const { input } = require('gpu.js');
// Faster upload of pre-flattened arrays
const value = input(flattenedArray, [width, height, depth]);

GPU.js pipelines execute using different strategies depending on the backend and the complexity of the plan:

Generic Executor

Used as a correctness reference and fallback. It executes steps sequentially through existing kernel machinery.

• CPU: Uses plain arrays.
• WebGL/WebGL2: Uses textures end-to-end.
• WebGPU: Uses buffer handles.

WebAssembly (Wasm) Fused Executor

Designed for high performance by keeping intermediates in Wasm memory.

• Sync path: All steps compile over a single Wasm memory layout [pipeline args | plan buffers]. Intermediates never leave Wasm memory.
• Threaded path: Uses workers with Atomics-based barriers between steps over shared memory. Falls back to sync-fused if threads are unavailable.

WebGPU Fused Executor ('fused-encoder')

Compiles the plan into a single command encoder.

• Every step is recorded as a compute pass into one command encoder.
• Uses persistent STORAGE buffers on the device with static alternating bind groups for ping-ponging.
• Performs one queue.submit and one mapAsync readback per pipeline call.

Degradation: If a plan contains elements that cannot be fused (e.g., GPU-resident handle arguments or certain vector intermediates), the system will degrade to the Generic Executor and provide a fallbackReason.

Pipelining allows values to be sent directly from one kernel to another via a texture, resulting in extremely fast computing.

How to enable:

• Use the kernel setting pipeline: true in createKernel.
• Or call kernel.setPipeline(true) on an existing kernel.

Memory Management:

• By default (immutable: false), kernel results are reused.
• If you want to keep results around, use immutable: true.
• Cleanup: In GPU mode, use texture.delete() to release memory. If you want to keep the texture in memory but reset its values to zero, use texture.clear().
• Cloning: When in pipeline mode, you can use texture.clone() to create a copy of the output.

const kernel1 = gpu.createKernel(function(v) {
    return v[this.thread.x];
})
  .setPipeline(true)
  .setOutput([100]);

const kernel2 = gpu.createKernel(function(v) {
    return v[this.thread.x];
})
  .setOutput([100]);

const result1 = kernel1(array);
// result1 is a Texture
const result2 = kernel2(result1);
// result2 is a Float32Array

Since you cannot set breakpoints on the GPU, the best way to inspect values is to return them early. By modifying your kernel to return a specific intermediate value, you can see exactly what the GPU is calculating at that step.

For graphical kernels, you can also use this.color(r, g, b) to visually debug by coloring pixels based on specific conditions.

// Debugging values by returning early
const gpu = new GPU({ mode: 'cpu' });
const kernel = gpu.createKernel(function(arg1, time) {
  const x = this.thread.x * time;
  return x; // Return x to see its value instead of continuing
  const v = arg1[this.thread.y][x];
  return v;
}).setOutput([100, 100]);

// Debugging graphical output
const gpu = new GPU({ mode: 'cpu' });
const kernel = gpu.createKernel(function(arg1, time) {
  const x = this.thread.x * time;
  if (x < 4 || x > 2) {
    this.color(1, 0, 0); // Red
    return;
  }
  return arg1[this.thread.y][x];
}, { output: [100, 100], graphical: true });