Spark 3D Gaussian Splatting Renderer

Spark is a dynamic 3DGS renderer designed for THREE.js and WebGL2. Key capabilities include:

• Pipeline Integration: Fuses splat and mesh-based objects within the THREE.js rendering pipeline.
• Portability: Targets 98%+ WebGL2 support, working across desktop, mobile, and WebXR.
• Performance: Optimized for fast rendering even on low-powered mobile devices.
• Multi-Object Rendering: Renders multiple splat objects together with correct occlusion sorting.
• Format Support: Supports major splat file formats including .PLY (including compressed), .SPZ, .SPLAT, and .KSPLAT.
• Dynamic Control: Supports real-time color editing, displacement, skeletal animation, and simultaneous multiple viewpoints.
• GPU Programmability: Includes a shader graph system for dynamic splat creation and editing via Dynos.

Spark is an advanced 3D Gaussian Splatting renderer designed for integration with THREE.js. It allows developers to render high-quality splats within a standard 3D scene alongside other meshes.

Key capabilities include:

• Seamless Integration: Works with existing THREE.js scenes, meshes, and other splats.
• Performance: Optimized for fast rendering across all devices.
• Programmability: Supports programmable dynamic splat effects.
• Wide Format Support: Compatible with .ply, .sogs (Self-Organizing Gaussians), .spz, .splat, and .ksplat formats.

The dyno shader graph system allows you to create custom computation graphs using JavaScript (and optionally GLSL) that are compiled to GLSL and run on the GPU. It uses TypeScript to ensure type safety and static validation of the GPU computation graph.

Key concepts:

• Dyno blocks: Function blocks with multiple typed inputs and outputs.
• DynoVal<T>: Values passed between blocks, where T is a DynoType representing a GLSL type.
• GLSL Integration: You can compose existing dyno functions or write raw GLSL code. Note that regular JavaScript operators (like +) cannot be used inside a dyno graph; you must use dyno helper functions (e.g., dyno.add(x, y)).

To maintain 60+ FPS, consider the following recommended splat budgets based on the target hardware:

• Quest 3: 1 million splats or less (avoid high concentration in small areas).
• Android phone: 1-2 million splats.
• iPhone: 1-3 million splats.
• Computer: 1-5 million splats (up to 10-20+ million on high-end desktops).

Note that high concentrations of splats in a small area (e.g., 500K splats from a Trellis object at a small screen scale) can bottleneck the GPU's rendering and blending capabilities even if the total count is below the budget.

Spark allows you to add 3D Gaussian Splatting (3DGS) to your existing THREE.js scenes. By creating SplatMesh objects, you can render splats alongside traditional triangle-based meshes using your standard render(scene, camera) call.

SplatMesh objects derive from THREE.Object3D, meaning they can be translated, rotated, placed anywhere in the scene hierarchy, and animated by adjusting their values each frame. You can instantiate a SplatMesh using a url parameter pointing to a splat file or by creating splats individually.

When adding multiple SplatEditSdf shapes to a SplatEdit, their values are blended using an exponential "softmax" function. This blending is commutative, meaning the order of addition does not matter.

To control the appearance of the transitions:

• Increase SplatEdit.sdfSmooth to control the scale of blending between different SDF shapes.
• Increase SplatEdit.softEdge to control the scale of the soft inside-outside boundaries of each shape.

Displacement (XYZ) is blended in the same way as RGBA, allowing for complex, smooth deformations like rippling effects.

Spark uses a SparkRenderer object within your THREE.Scene to perform Gaussian Splat rendering. To use it, create an instance of SparkRenderer and add it to your scene (typically at the root). This object allows you to control rendering parameters like Level-of-Detail (LoD).

Note: Your THREE.WebGLRenderer should be created with antialias: false to maximize performance, as WebGL anti-aliasing does not improve Gaussian Splatting results.

const spark = new SparkRenderer({
  renderer: myThreeJsWebGlRenderer,
});
const scene = new THREE.Scene();
scene.add(spark);

To build Spark from source, you must have Rust installed on your machine. Follow these steps to build the WASM components and run the local development server:

```shell
npm install
npm run build:wasm
npm run dev

After running npm run dev, the examples will be available at http://localhost:8080/.

Spark can automatically detect and load .ply (including compressed SuperSplat/gsplat variants and plain point clouds) and .spz (Niantic compressed format) based on file contents. You can instantiate a SplatMesh directly using a URL pointing to these files.

// Load and create SplatMesh in one go
const splats = new SplatMesh({ url: "./butterfly.ply" });
scene.add(splats);

// Auto-detection works even without file extensions
scene.add(new SplatMesh({ url: "plyBin/0123456789abcdef" }));
scene.add(new SplatMesh({ url: "spzBin/fedcba9876543210" }));

The Spark dyno system provides a standard library of Dyno blocks covering GLSL ES 3.0 functions. You can use two different syntaxes for these functions:

1. PascalCase Classes: Using the new keyword with the class name.
2. camelCase Helpers: Using ergonomic helper functions that internally instantiate the corresponding class.

Both approaches are functionally equivalent.

To use Spark in a Node.js or bundler-based project, install the package using npm.

npm install @sparkjsdev/spark