DOCSEARCHbyAlgolia
Submit Docs

Shader Knowledge for VRChat

repository·main·Indexed 42 days ago

https://github.com/pema99/shader-knowledge

A technical knowledge base documenting advanced and obscure shader development techniques for VRChat. Covers stateful shaders using camera loops, transferring shader data to Udon, GPU instancing for object tracking, geometry shaders, raymarching, and depth texture reconstruction.

Tokens
11.3K
Snippets
41
Records
56
Agent score
54%

What's inside Shader Knowledge

  1. Overview of Shader Knowledge for VRChat

    main

    This repository serves as a collection of technical documentation focused on obscure shader development knowledge specifically tailored for VRChat. It covers advanced topics such as stateful shaders using camera loops, transferring shader data to Udon, and leveraging GPU instancing for object tracking.

    Note: This repository is no longer actively maintained, though the information is generally considered correct. Users are encouraged to contribute via Pull Requests or Issues if they find inaccuracies or wish to add new topics.

  2. Implement MRT camera loops with double-buffering

    main

    MRT can be used to simulate multiple consecutive camera loops in a single pass. To prevent feedback loops and ensure stability, the camera loops must be double-buffered.

    Implementation Pattern

    1. Define Buffers: For $N$ simulated loops, create $2 imes N$ RenderTextures (one set for the main loop data, one set for the double-buffer/copy pass).
    2. Assign Cameras: Use SetTargetBuffers on both the LoopCamera and the DBCamera (Double Buffer Camera) to point them to their respective sets of RenderTextures.
    3. Shader Logic:
      • The Main Loop Shader reads from the previous loop's textures and writes results to the MRT targets.
      • The Double-Buffering Shader performs a simple data copy from the main textures to the double-buffer textures.
    4. Criss-Cross Textures: To complete the loop, feed the output textures of the first camera into the quad associated with the second camera, and vice-versa.
    // Example setup for 3 simulated loops using 2 cameras
void Start()
{
    // LoopCamera writes to RT0, RT1, RT2
    LoopCamera.SetTargetBuffers(
        new RenderBuffer[] { RT0.colorBuffer, RT1.colorBuffer, RT2.colorBuffer }, 
        RT0.depthBuffer);

    // DBCamera copies RT0, RT1, RT2 into the double buffers
    DBCamera.SetTargetBuffers(
        new RenderBuffer[] { RT0_DB.colorBuffer, RT1_DB.colorBuffer, RT2_DB.colorBuffer }, 
        RT0_DB.depthBuffer);
}

  3. What are camera loops and when to use them

    main

    Camera loops are a technique for creating stateful shaders—shaders that can store state and base new calculations on previous states. This is useful for implementing GPU particles, fluid simulations, or shader-based games.

    Key Characteristics:

    • Statefulness: You can treat each pixel in a RenderTexture as a memory cell that can be read from and written to.
    • Avatar Compatibility: Unlike Custom Render Textures (CRTs), camera loops work well on avatars.
    • Visibility Caveat: Camera loops are only visible to friends, regardless of the user's safety settings.

  4. Performance note: Geometry shaders and VRChat polygon counts

    main
    In VRChat, geometry shaders can be used to 'spoof' performance ranks. Any additional geometry generated dynamically within a geometry shader does not count towards the avatar's total polygon count. Use this capability responsibly.

  5. Optimize depth writing with SV_DepthLessEqual

    main

    Using SV_Depth disables early Z testing, which can degrade performance. To write depth while preserving early Z testing, use the SV_DepthLessEqual semantic. This semantic only writes the depth if the value is less than or equal to the value determined by the rasterizer.

    Requirement: This requires a modern shading model. You must add #pragma target 5.0 to your shader to use this semantic.

    #pragma target 5.0

float4 frag (v2f i, out float depth : SV_DepthLessEqual) : SV_Target
{
    // ... depth calculation
}

  6. Use geometry shaders for blitting to camera loops and RenderTextures

    main

    Geometry shaders can bypass the limitation of camera loops where you can only write to the pixel currently being shaded. By spawning geometry (such as pixel-sized quads) in front of the camera's capture quad, you can effectively perform arbitrary writes to a RenderTexture.

    Common Use Cases:

    • Arbitrary Writes: Spawning quads with specific colors to 'blit' data into a camera loop buffer.
    • Data Storage: Creating a grid of small quads where each quad's color encodes the vertex position of a skinned mesh, then capturing that grid with a camera into a RenderTexture for use in other shaders.
    • Screen Access: Drawing data directly into the camera's clipspace. If combined with a GrabPass, this data can be made globally accessible.

  7. Handle Skinned Mesh Renderer 'Update When Offscreen' in VRChat

    main
    In VRChat, the 'Update When Offscreen' setting is forced ON for local avatars and forced OFF for remote avatars. This can cause culling issues if geometry extends beyond the bounding box. You can override this behavior using an animation targeting the setting.

  9. Performance considerations for reading shader data

    main
    Reading pixel data from a RenderTexture into Udon via ReadPixels causes significant CPU overhead. This technique should only be used when there is a specific requirement that cannot be met by other means. For modern implementations, use the VRChat Async GPU Readback API instead.

  10. Prevent light array pollution with a dummy ForwardAdd pass

    main

    The unity_4LightPos arrays may contain pixel lights if the shader lacks an add pass. To ensure they only contain vertex lights, add a dummy ForwardAdd pass that does nothing.

    Pass
{
    Tags { "LightMode"="ForwardAdd" }
    CGPROGRAM
    #pragma vertex empty
    #pragma fragment empty
    void empty(){}
    ENDCG
}

  11. Write depth for raymarched shaders using SV_Depth

    main

    To allow raymarched effects to interact correctly with other Unity renderers (like shadows or depth-based effects), you must write depth from the fragment shader using the SV_Depth semantic.

    1. Add an out float depth : SV_Depth parameter to your fragment function.
    2. Calculate the hit_position in world space.
    3. Transform the hit_position into clip space using UNITY_MATRIX_VP.
    4. Output the depth as clip_pos.z / clip_pos.w.
    float4 frag (v2f i, out float depth : SV_Depth) : SV_Target
{
    // ... calculate hit_position in world space
    float3 hit_position = ray_origin + ray_distance * ray_direction;

    // Output depth
    float4 clip_pos = mul(UNITY_MATRIX_VP, float4(hit_position, 1.0));
    depth = clip_pos.z / clip_pos.w;

    return float4(1, 1, 1, 1);
}

  12. Handle ray origin when camera is outside the shaded object

    main

    If the camera is outside the geometry being shaded (e.g., a cube), setting the ray origin directly to the camera position can cause visual artifacts. To fix this, ensure the ray starts from the object's surface when looking at the front face.

    1. Set Cull Off in your SubShader to ensure backfaces are processed.
    2. Use the VFACE semantic in your fragment shader to detect the face orientation.
    3. If facing > 0 (front face), set the ray_origin to the vert_position instead of the camera position.
    SubShader
{
    Cull Off
    ...
}

float4 frag (v2f i, float facing : VFACE) : SV_Target
{
    float3 ray_origin = i.ray_origin;

    if (facing > 0) // if front face, move ray origin to vertex pos
        ray_origin = i.vert_position;

    // ... continue raymarching
}