GauStudio

repository·master·Indexed 23 days ago

https://github.com/gap-lab-cuhk-sz/gaustudio

A modular framework designed to accelerate research and development in 3D Gaussian Splatting (3DGS) and its applications. It provides tools for mesh extraction, rendering, and dataset management, including a unified interface for loading various 3D datasets via gaustudio.datasets.make. The framework includes an extended differential Gaussian rasterization engine supporting analytical gradients for rendered opacity and depth, as well as median depth inference.

Tokens
17.1K
Snippets
31
Records
77
Agent score
82%

What's inside GauStudio

  1. Overview of OpenGL Mathematics (GLM)

    master

    GLM is a header-only C++ mathematics library designed for graphics software. It is based on the OpenGL Shading Language (GLSL) specifications, meaning it uses the same naming conventions and functionality as GLSL. This allows developers familiar with GLSL to use the same logic in C++.

    Key features include:

    • GLSL Compatibility: Classes and functions mirror GLSL.
    • Extension System: Provides additional capabilities like matrix transformations, quaternions, data packing, random numbers, and noise.
    • Versatility: While optimized for OpenGL, it is suitable for software rendering (raytracing/rasterization), image processing, and physics simulations.
    • Header-only: Easy to integrate as it requires no compiled binaries.
  2. Overview of Differential Gaussian Rasterization for GauStudio

    master

    This software serves as the rasterization engine for GauStudio. It is an extended version of the original Differential Gaussian Rasterization used in '3D Gaussian Splatting for Real-Time Rendering of Radiance Fields'.

    Key supported features include:

    • Analytical gradient for rendered opacity.
    • Inference of median depth (via JonathonLuiten).
    • Analytical gradient for median depth.
    • Analytical gradient for rendered depth (via ingra14m and slothfulxtx).
  3. How to use GLM extensions

    master

    GLM extends its core GLSL feature set through dedicated extension headers. To use an extension, include its specific header file. Once included, the extension's features are added to the glm namespace. Including an extension also automatically includes all its dependent core functionalities and other required extensions.

    #include <glm/glm.hpp>
    #include <glm/gtc/matrix_transform.hpp>
    
    int foo()
    {
        glm::vec4 Position = glm::vec4(glm:: vec3(0.0f), 1.0f);
        glm::mat4 Model = glm::translate(glm::mat4(1.0f), glm::vec3(1.0f));
    
        glm::vec4 Transformed = Model * Position;
        ... 
    
        return 0;
    }
  4. Handle GLM precision qualifiers

    master

    GLM supports GLSL precision qualifiers through type prefixes instead of qualifiers. By default, all types use high precision. You can use these to trade precision for performance.

    Available prefixes:

    • lowp_ (e.g., lowp_vec4)
    • mediump_ (e.g., mediump_vec4)
    • highp_ (e.g., highp_vec4)

    Example usage:

    #include <glm/glm.hpp>
    
    ivec3 foo(const vec4 & v)
    {
        highp_vec4 a = v;
        medium_vec4 b = a;
        lowp_ivec3 c = glm::ivec3(b);
        return c;
    }
  5. Enable GLSL-style swizzle operators

    master

    To enable swizzling (e.g., vec.xyz(), vec.rgba), define GLM_FORCE_SWIZZLE before including GLM headers.

    Warning: Enabling swizzling can significantly increase binary size and compilation time.

    Implementation Modes:

    1. Standard C++98: Uses member functions (e.g., ColorRGBA.bgr()). These return copies and cannot be used as L-values (you cannot assign to them).
    2. Language Extensions (Visual C++, GCC, Clang): Uses anonymous struct/union extensions to allow both L-value and R-value swizzling (e.g., ColorRGBA.bgra = ColorRGBA.rgba;).

    Note: When using extension-based swizzling, the resulting types are not standard vector types and must be explicitly converted via a constructor vec4(Color.rgba) or operator Color.rgba() before being passed to functions like clamp().

    #define GLM_FORCE_SWIZZLE
    #include <glm/glm.hpp>
    
    // C++98 style (R-value only)
    glm::vec3 const ColorBGR = ColorRGBA.bgr();
    
    // Extension style (L-value and R-value)
    // Only guaranteed to work with Visual C++!
    glm::vec4 ColorBGRA = ColorRGBA.bgra;
  6. Use GLM extensions and features

    master

    GLM provides core functionality and various extensions (prefixed with GLM_EXT_, GLM_GTX_, or GLM_GTC_).

    Common Extensions

    • Matrix Operations: GLM_EXT_matrix_integer, GLM_EXT_matrix_clip_space, GLM_GTX_matrix_factorisation.
    • Vector Operations: GLM_EXT_vector_reciprocal, GLM_EXT_vector_intX, GLM_EXT_vector_uintX, GLM_EXT_vector_relational.
    • Quaternion Support: GLM_GTC_quaternion, GLM_GTX_quaternion.
    • Color & Packing: GLM_GTC_packing, GLM_GTX_color_encoding, GLM_GTX_texture.

    Key Functions and Improvements

    • Math Utilities: glm::clamp, glm::repeat, glm::mirrorClamp, glm::mirrorRepeat (added in 0.9.9.8).
    • Rounding: glm::iround and glm::uround (added in 0.9.9.9).
    • Matrix Mixing: glm::mix implementation for matrices (added in 0.9.9.4).
    • SIMD/Hardware Support: Support for Neon (ARM) and SYCL (added in 0.9.9.6).
  7. Extract a mesh from 3DGS data

    master

    To extract a mesh from 3D Gaussian Splatting (3DGS) output, use the gs-extract-mesh command.

    Input Data Requirements

    The input directory must follow the structure used by methods like 3DGS, mip-splatting, or GaussianPro:

    • cameras.json (required)
    • point_cloud/iteration_xxxx/point_cloud.ply (required)

    Usage

    gs-extract-mesh -m <input_output_dir> -o <output_mesh_path>

    Example:

    gs-extract-mesh -m ./data/result -o ./output/mesh_result
    gs-extract-mesh -m ./data/1750250955326095360_data/result -o ./output/1750250955326095360_data
  8. Optimize build times using separated core headers

    master

    To minimize compilation times, avoid global headers and instead include only the specific core features you need. GLM provides individual headers for vectors, matrices, and various mathematical function groups based on the GLSL specification.

    // Include GLM core features
    #include <glm/vec2.hpp>           // vec2
    #include <glm/vec3.hpp>           // vec3
    #include <glm/mat4x4.hpp>         // mat4
    #include <glm/trigonometric.hpp>  // radians
    
    // Include GLM extension
    #include <glm/ext/matrix_transform.hpp> // perspective, translate, rotate
    
    glm::mat4 transform(glm::vec2 const& Orientation, glm::vec3 const& Translate, glm::vec3 const& Up)
    {
        glm::mat4 Proj = glm::perspective(glm::radians(45.f), 1.33f, 0.1f, 10.f);
        glm::mat4 ViewTranslate = glm::translate(glm::mat4(1.f), Translate);
        glm::mat4 ViewRotateX = glm::rotate(ViewTranslate, Orientation.y, Up);
        glm::mat4 View = glm::rotate(ViewRotateX, Orientation.x, Up);
        glm::mat4 Model = glm::mat4(1.0f);
        return Proj * View * Model;
    }
  9. Use GLM with global headers

    master

    GLM is a header-only library. For convenience, you can include the main <glm/glm.hpp> header to access core GLSL mathematics functionality, and <glm/ext.hpp> to access extensions like perspective, translate, and rotate.

    Warning: Including these global headers can significantly increase compilation times because they pull in a large amount of code. Use them primarily for small projects or when build speed is not a priority.

    #include <glm/glm.hpp> // vec2, vec3, mat4, radians
    #include <glm/ext.hpp> // perspective, translate, rotate
    
    glm::mat4 transform(glm::vec2 const& Orientation, glm::vec3 const& Translate, glm::vec3 const& Up)
    {
        glm::mat4 Proj = glm::perspective(glm::radians(45.f), 1.33f, 0.1f, 10.f);
        glm::mat4 ViewTranslate = glm::translate(glm::mat4(1.f), Translate);
        glm::mat4 ViewRotateX = glm::rotate(ViewTranslate, Orientation.y, Up);
        glm::mat4 View = glm::rotate(ViewRotateX, Orientation.x, Up);
        glm::mat4 Model = glm::mat4(1.0f);
        return Proj * View * Model;
    }
  10. GLM Compiler Support and Requirements

    master

    GLM is a platform-independent, header-only library written in C++98 (with C++11 support). It has no external dependencies.

    Officially supported compilers include:

    • GCC: 4.7 and higher
    • Clang: 3.4 and higher
    • Apple Clang: 6.0 and higher
    • Visual C++: 2013 and higher
    • Intel C++ Compiler XE: 2013 and higher
    • CUDA: 9.0 and higher (experimental)
    • SYCL: Experimental (tested with ComputeCpp implementation)
    • Any C++11 compiler