DOCSEARCHbyAlgolia
Submit Docs

Diligent FX

repository·master·Indexed 19 days ago

https://github.com/diligentgraphics/diligentfx

A high-level rendering framework for the Diligent Engine. It provides advanced rendering features including PBR, a suite of post-processing effects, and the ShadowMapManager for managing shadow map textures, cascades, and filtering (PCF, VSM, EVSM). It also includes Hydrogent, an implementation of the Hydra rendering API that integrates with OpenUSD.

Tokens
21.9K
Snippets
31
Records
68
Agent score
65%

What's inside diligentfx

  1. Overview of Diligent FX components

    master

    Diligent FX is a high-level rendering framework built on top of the Diligent Engine. It provides several specialized rendering components and post-processing effects:

    Core Rendering Components

    • GLTF2.0 Loader & PBR Renderer: Includes a GLTF 2.0 loader and a Physically-Based Renderer (PBR) with image-based lighting.
    • Hydrogent: An implementation of the Hydra rendering API for Diligent Engine.
    • Shadows: A component for handling shadow rendering.

    Post-processing Effects

    • Screen-Space Reflections (SSR)
    • Screen-Space Ambient Occlusion (SSAO)
    • Depth of Field (DoF)
    • Bloom
    • Epipolar Light Scattering
    • Temporal Anti-Aliasing (TAA)
    • Tone mapping shader utilities

  2. Overview of Screen Space Reflections (SSR) in Diligent FX

    master

    Diligent FX provides a Screen Space Reflections (SSR) implementation designed to meet specific performance and compatibility requirements. The implementation is based on AMD's Stochastic Screen Space Reflections (SSSR) algorithm but has been adapted to support environments like WebGL that lack compute shader support.

    Key Features & Requirements:

    • WebGL Compatibility: Optimized to work in environments without compute shaders.
    • Rough Surface Support: Capable of rendering reflections on non-perfectly smooth surfaces.
    • Performance Target: Designed to execute in under 2ms at Full HD resolution on hardware equivalent to an NVIDIA RTX 2070.

  4. How projection matrices are constructed in Radient

    master

    Radient uses a hybrid approach to reconcile its scene camera convention with Diligent's projection helpers:

    1. Radient Camera Space: Uses local -Z as forward.
    2. Diligent Camera Space: Uses left-handed space where positive Z is forward.

    To bridge these, the projection matrix (mProj) includes a camera-space adapter. This results in the following matrix properties:

    • mView: The true inverse of the camera world transform.
    • mViewInv: The camera world transform.
    • mProj: Includes the conversion from Radient camera space to Diligent camera space.

    Important for Renderer/Post-processing Developers: Treat the camera world transform as a standard Radient scene transform. Do not apply an additional Z-flip outside of the Radient camera projection path.

  5. Understand the Screen Space Reflections (SSR) algorithm structure

    master

    The Diligent FX SSR implementation is a multi-stage post-processing effect divided into three main phases:

    1. Preparation for ray tracing: Generates necessary resources including blue noise textures, a hierarchical depth buffer (Hi-Z), and a stencil mask for roughness extraction.
    2. Ray Tracing: Performs hierarchical ray marching to solve the specular part of the rendering equation using a Split-Sum-Approximation.
    3. Denoising: A three-step process to clean up stochastic noise:
      • Spatial reconstruction: Accumulates radiance from nearby points.
      • Temporal accumulation: Uses temporal coherence and specialized reprojection (accounting for reflected object depth) to accumulate samples over time.
      • Cross-bilateral filtering: A final cleanup pass using depth and normals to form the range kernel.

  6. How Epipolar Light Scattering works

    master

    Epipolar Light Scattering is a post-processing effect that renders high-quality light scattering by placing expensive ray-marching samples along epipolar lines (lines starting at the light source) and interpolating radiance between them. This approach is significantly more efficient than brute-force ray marching, which performs expensive calculations for every screen pixel.

    Key performance/quality trade-offs:

    • Epipolar Sampling: Uses lines to reduce sample count.
    • Brute Force: Can be used as a quality reference by performing ray marching for every pixel.
    • Refinement: Discontinuities can be detected using either uiRefinementCriterion set to depth difference or scattering difference (the latter is generally preferred).

  7. Prepare resources for SSR ray tracing

    master

    Before ray tracing can occur, several resources must be generated:

    • Blue Noise Texture: A 128×128 animated blue noise texture is used to drive stochastic sampling of the specular lobe. Diligent FX generates two textures simultaneously to prevent correlation between SSR and SSAO.
    • Hierarchical Depth Buffer (Hi-Z): A mip chain where each level contains the minimum (or maximum for reserved depth) of the 2×2 area of the previous level. This is used to accelerate ray marching by skipping empty space.
    • Stencil Mask and Roughness Extraction:
      • A stencil mask is used to mark pixels that participate in ray tracing (where roughness is below a RoughnessThreshold).
      • A separate render target is used to store roughness values to simplify sampling in later stages.
      • Pixels meeting the threshold are marked with 0xFF in the stencil buffer.

  8. Apply SSAO to Diffuse Radiance

    master

    Once the SSAO texture is obtained via GetAmbientOcclusionSRV, apply it to your lighting equation. A common approach is to modulate the diffuse component:

    $$\text{DiffuseRadiance} = \sum_{i=1}^{n} (1 - F) \times \text{Diffuse} \times \text{SSAO} \times \text{IrradianceMap}_i$$

    Where $F$ is the Fresnel Schlick coefficient (e.g., using FresnelSchlickWithRoughness).

  9. Denoise SSR results

    master

    Because the ray tracing stage is stochastic, the output is noisy and requires a three-stage denoising pipeline:

    1. Spatial Reconstruction: Accumulates incoming radiance from nearby points based on the assumption that closely located surfaces share visibility. It calculates variance and the maximum ray length (needed for temporal parallax correction) and stores them in separate textures.
    2. Temporal Accumulation: Accumulates the current frame with previous frames. Unlike standard TAA, this uses a specialized reprojection method that accounts for the fact that reflected objects move according to their own depth, not the depth of the reflecting surface. It uses ray length for parallax correction.
    3. Cross-bilateral Filtering: A final cleanup pass. It uses the variance from the spatial reconstruction stage to determine the spatial kernel ($\sigma$) and uses the depth buffer and normals buffer to form the range kernel ($G_r$), following the SVGF algorithm approach.

  10. Understand Radient light emission conventions

    master

    Radient follows the glTF and OpenUSD conventions for oriented lights. The emission direction depends on the light type:

    • Directional lights: Emit along their local negative Z axis. Orientation matters; position does not affect direct lighting.
    • Spot lights: Emit along their local negative Z axis. Both position (cone origin) and orientation (cone axis) matter.
    • Point lights: Emit from their world-space position. Orientation does not affect direct lighting.

    To calculate the world-space direction for directional or spot lights using Radient's row-vector matrix convention, use the transformed local -Z axis.

    const RadientFloat4 LocalZ = WorldMatrix.GetRow(2);
const float3 Direction = normalize(float3{-LocalZ.x, -LocalZ.y, -LocalZ.z});

  11. How TAA handles jitter and camera updates

    master

    TAA requires that each frame is rendered with a sub-pixel offset (jitter) relative to the pixel grid.

    To implement this, you must:

    1. Retrieve the jitter offset from m_TAA->GetJitterOffset().
    2. Modify your projection matrix by injecting the jitter into the third row (e.g., Result[2][0] = Jitter.x; Result[2][1] = Jitter.y;).
    3. Update the HLSL::CameraAttribs structure with the jittered matrices (View, Proj, ViewProj, and their inverses) and the f2Jitter field.
    auto ComputeProjJitterMatrix = [&](const float4x4& ProjMatrix, float2 Jitter) -> float4x4 {
    float4x4 Result = ProjMatrix;
    Result[2][0]    = Jitter.x;
    Result[2][1]    = Jitter.y;
    return Result;
};

const float2 Jitter = m_TAA->GetJitterOffset();
const float4x4 CameraView     = m_Camera.GetViewMatrix();
const float4x4 CameraProj     = ComputeProjJitterMatrix(GetAdjustedProjectionMatrix(YFov, ZNear, ZFar), Jitter);
const float4x4 CameraViewProj = CameraView * CameraProj;

// Update HLSL::CameraAttribs
CurrCamAttribs.mViewT         = CameraView.Transpose();
CurrCamAttribs.mProjT         = CameraProj.Transpose();
CurrCamAttribs.mViewProjT     = CameraViewProj.Transpose();
CurrCamAttribs.mViewInvT      = CameraView.Inverse().Transpose();
CurrCamAttribs.mProjInvT      = CameraProj.Inverse().Transpose();
CurrCamAttribs.mViewProjInvT  = CameraView.Inverse().Transpose();
CurrCamAttribs.f2Jitter.x     = Jitter.x;
CurrCamAttribs.f2Jitter.y     = Jitter.y;

  12. How SSR Ray Tracing works

    master

    The ray tracing stage solves the specular part of the rendering equation using the Split-Sum-Approximation.

    Key Implementation Details:

    • Half-Vector Generation: Uses the VNDF (Visible Normal Distribution Function) method by Eric Heitz to ensure generated rays do not fall below the horizon.
    • Hierarchical Ray Marching: The algorithm traverses the Hi-Z depth buffer mip chain. It starts at mip 0, drops to lower resolutions to skip empty space, and climbs back up to higher resolutions when a collision is detected.
    • Radiance Sampling: If a ray intersects the scene, radiance is sampled from the ColorBuffer. If no intersection occurs, the algorithm does not fallback to an environment map automatically; instead, it writes a confidence value (approx. 1 for hit, 0 for miss) into the alpha channel of the resulting texture. The user is responsible for interpolating between the SSR result and the Environment Map using this confidence value.
    • GGX Bias: A parameter is available to reduce variance in the specular reflections.