encase

repository·main·Indexed 42 days ago

https://github.com/teoxoy/encase

A Rust library for laying out data into GPU buffers while ensuring compliance with WGSL (WebGPU Shading Language) memory layout requirements. It provides a trait-based architecture via `ShaderType` and buffer wrappers like `UniformBuffer`, `StorageBuffer`, and `DynamicStorageBuffer` to automate padding and alignment for GPU-bound data. Version 0.12.2.

Tokens
9.8K
Snippets
29
Records
37
Agent score
53%

What's inside encase

  1. How encase manages GPU memory layout

    main

    Encase provides a mechanism to lay out Rust data into GPU buffers while ensuring compliance with WGSL (WebGPU Shading Language) memory layout requirements. It uses a trait-based architecture to ensure that data structures are correctly aligned and padded for GPU consumption at compile time.

    Core Abstractions

    • ShaderType: The central trait containing metadata about a type's layout.
    • WriteInto, ReadFrom, and CreateFrom: Traits that define how a type can be serialized into, deserialized from, or instantiated from a buffer.
    • Buffer Wrappers: Structs that wrap raw buffers (implementing BufferRef or BufferMut) to facilitate high-level operations:
      • UniformBuffer / DynamicUniformBuffer (for uniform data)
      • StorageBuffer / DynamicStorageBuffer (for storage data)
  2. What is a `VectorScalar`?

    main
    A VectorScalar is a marker trait for types that can serve as the individual components of a vector in a shader (e.g., f32, u32, i32). To be a VectorScalar, a type must also implement crate::ShaderSize. This ensures that the vector's total size and alignment can be correctly calculated for buffer layouts.
  3. Handle runtime-sized arrays and structs

    main

    For types that are not fixed-footprint (e.g., WGSL runtime-sized arrays or structs containing them), you must account for their dynamic size.

    • CalculateSizeFor: Use this trait to calculate the required buffer size based on the number of elements in the runtime-sized array.
    • RuntimeSizedArray: A trait providing a .len() method to retrieve the current number of elements in a runtime-sized array.

    When calculating buffer requirements for these types, use calculate_size_for(nr_of_el) where nr_of_el is the number of elements currently in the array.

    // Conceptual usage for a runtime-sized type
    let num_elements = my_runtime_array.len() as u64;
    let required_size = <MyRuntimeType as CalculateSizeFor>::calculate_size_for(num_elements);
  4. Use `ArrayLength` for robust runtime-sized arrays in shaders

    main

    The ArrayLength type is a helper designed to be used with the #[derive(ShaderType)] macro to solve limitations with the WGSL arrayLength() function.

    In WGSL, arrayLength() can be unreliable because it might return a minimum of 1 due to minBindingSize calculations, or it might include padding at the end of a buffer as extra elements.

    When you include ArrayLength as a field in a struct using #[derive(ShaderType)]:

    • On Write: It automatically writes the actual length of the contained runtime-sized array as an u32 to the buffer.
    • On Read: It reads that u32 value (representing the length) and ensures that a maximum of that many elements are read from the buffer, preventing the consumption of trailing padding as array elements.

    ArrayLength is treated as a 4-byte u32 in the shader.

    #[derive(ShaderType)]
    struct MyStruct {
        len: ArrayLength,
        data: Vec<MyElement>, // This is the runtime-sized array
    }
  5. Array layout and metadata for GPU buffers

    main

    When using arrays [T; N] with encase, the library automatically calculates the memory layout required for GPU compatibility. This includes handling alignment and padding between elements to ensure the array matches shader expectations.

    Key concepts for arrays:

    • Stride: The distance in bytes between the start of one element and the start of the next. This is calculated by rounding up the element size to the nearest alignment boundary of the type T.
    • Element Padding (el_padding): The number of bytes added after each element to satisfy alignment requirements.
    • Uniform Compatibility: The library includes internal assertions to ensure that the array stride is a multiple of the required minimum alignment for uniform buffers.

    Note: 0-sized arrays are not supported.

  6. Implement `ShaderType` for structs using `#[derive(ShaderType)]`

    main

    To prepare Rust structs for use in GPU buffers (like WGSL), derive the ShaderType trait. This allows the encase crate to calculate the correct memory layout, alignment, and size required by shaders.

    Field Attributes

    You can use the #[shader(...)] attribute on struct fields to control layout:

    • #[shader(align(X))]: Sets the alignment to a power-of-2 u32 literal (e.g., #[shader(align(16))]).
    • #[shader(size(X))]: Sets the size to a u32 literal.
    • #[shader(size(runtime))]: Must be attached to the last field of a struct to denote a runtime-sized array.

    Note on Generics

    Attributes like align and size only work on fields whose types do not contain generic type parameters.

    Examples

    Simple Struct:

    #[derive(ShaderType)]
    struct AffineTransform2D {
        matrix: test_impl::Mat2x2f,
        translate: test_impl::Vec2f,
    }

    Struct with Runtime-Sized Array: Use the ArrayLength type to explicitly handle the length of the runtime-sized array.

    #[derive(ShaderType)]
    struct Positions {
        length: ArrayLength,
        #[shader(size(runtime))]
        positions: Vec<test_impl::Vec2f>
    }

    Complex Generic Struct:

    #[derive(ShaderType)]
    struct Complex< 
        'a, 
        'b: 'a, 
        E: 'a + ShaderType + ShaderSize, 
        T: 'b + ShaderType + ShaderSize, 
        const N: usize 
    > {
        array: [&'a mut E; N],
        #[shader(size(runtime))]
        rts_array: &'a mut Vec<&'b T>,
    }
  7. Customize WGSL memory layout with `#[derive(ShaderType)]`

    main

    The encase_derive crate provides the ShaderType derive macro to automatically implement traits required for mapping Rust structs to WGSL (WebGPU Shading Language) memory layouts. This ensures that your Rust data structures match the expected alignment and padding rules of GPU buffers.

    You can use the #[shader(...)] attribute on struct fields to override default alignment or size behaviors.

    Supported Attributes

    • #[shader(align(N))]: Overrides the alignment of a field to N bytes. N must be a power of two.
    • #[shader(size(N))]: Overrides the size of a field to N bytes. This is a static size.
    • #[shader(size(runtime))]: Marks the last field of a struct as having a runtime-sized array. This is required if you want to use the ArrayLength type within the struct to represent the length of this array in the shader.

    Requirements

    • The struct must have named fields.
    • Only the last field can be marked with size(runtime).
    • If using ArrayLength for a field, the struct must also contain a field marked size(runtime).
    ```rust
    #[derive(encase::ShaderType)]
    struct MyUniform {
        #[shader(align(16))]
        data: f32,
        #[shader(size(runtime))]
        elements: encase::ArrayLength,
    }
  8. Implement `ShaderType` for runtime-sized arrays with `impl_rts_array!`

    main

    To make a collection (like Vec<T>) work as a runtime-sized array within a ShaderType struct, you must implement the ShaderType trait using the impl_rts_array! macro. This macro handles the complex logic of calculating strides, padding, and performing length-aware reads/writes.

    Requirements

    The type being implemented must:

    1. Implement Length (to provide the current size).
    2. Optionally implement Truncate (if the collection needs to be resized during reading).

    Macro Syntax

    impl_rts_array!($type; [generics]; using [capabilities])

    • $type: The type you are implementing for (e.g., Vec<T>).
    • $generics: (Optional) Any generic parameters.
    • using: (Optional) Specifies which capabilities to bridge from the underlying type:
      • len: Maps the collection's length method to the Length trait.
      • truncate: Maps the collection's truncation method to the Truncate trait.

    Examples

    For Vec<T> (supports resizing):

    impl_rts_array!(Vec<T>; using len truncate);

    For LinkedList<T> (read-only length):

    impl_rts_array!(LinkedList<T>; using len);
    // Example for a Vec of custom elements
    impl_rts_array!(Vec<MyElement>; using len truncate);
  9. Write multiple types to a DynamicStorageBuffer with alignment

    main

    When packing multiple different data types into a single buffer, use DynamicStorageBuffer::new_with_alignment to ensure each entry starts at a specific alignment boundary (e.g., 64 bytes).

    use encase::{ShaderType, DynamicStorageBuffer};
    
    let mut byte_buffer: Vec<u8> = Vec::new();
    
    // Create a buffer with a 64-byte alignment requirement
    let mut buffer = DynamicStorageBuffer::new_with_alignment(&mut byte_buffer, 64);
    
    // write() returns the offset where the data was written
    let offsets = [
        buffer.write(&[5.; 10]).unwrap(),
        buffer.write(&vec![3u32; 20]).unwrap(),
        buffer.write(&test_impl::Vec3f::ONE).unwrap(),
    ];
    
    // offsets will be [0, 64, 192]
    use encase::{ShaderType, DynamicStorageBuffer};
    
    let mut byte_buffer: Vec<u8> = Vec::new();
    
    let mut buffer = DynamicStorageBuffer::new_with_alignment(&mut byte_buffer, 64);
    let offsets = [
        buffer.write(&[5.; 10]).unwrap(),
        buffer.write(&vec![3u32; 20]).unwrap(),
        buffer.write(&test_impl::Vec3f::ONE).unwrap(),
    ];
    
    // write byte_buffer to GPU
    
    assert_eq!(offsets, [0, 64, 192]);
  10. Implement ShaderType for structs using derive

    main

    To make a custom struct compatible with GPU buffers, derive the ShaderType trait. This allows the struct to be used with encase buffer wrappers.

    use encase::{ShaderType, UniformBuffer};
    
    #[derive(ShaderType)]
    struct AffineTransform2D {
        matrix: test_impl::Mat2x2f,
        translate: test_impl::Vec2f,
    }
    
    let transform = AffineTransform2D {
        matrix: test_impl::Mat2x2f::IDENTITY,
        translate: test_impl::Vec2f::ZERO,
    };
    
    let mut buffer = UniformBuffer::new(Vec::<u8>::new());
    buffer.write(&transform).unwrap();
    let byte_buffer = buffer.into_inner();
    
    // byte_buffer is now ready to be sent to the GPU
    use encase::{ShaderType, UniformBuffer};
    
    #[derive(ShaderType)]
    struct AffineTransform2D {
        matrix: test_impl::Mat2x2f,
        translate: test_impl::Vec2f,
    }
    
    let transform = AffineTransform2D {
        matrix: test_impl::Mat2x2f::IDENTITY,
        translate: test_impl::Vec2f::ZERO,
    };
    
    let mut buffer = UniformBuffer::new(Vec::<u8>::new());
    buffer.write(&transform).unwrap();
    let byte_buffer = buffer.into_inner();
  11. Read data from a DynamicUniformBuffer at an offset

    main

    Use DynamicUniformBuffer to read specific types from a larger byte buffer at a given offset. This is useful when multiple uniform objects are packed into a single buffer.

    use encase::DynamicUniformBuffer;
    
    // byte_buffer represents data read back from the GPU
    let byte_buffer = [1u8; 256 + 8];
    
    let mut buffer = DynamicUniformBuffer::new(&byte_buffer);
    buffer.set_offset(256);
    let vector: test_impl::Vec2i = buffer.create().unwrap();
    use encase::DynamicUniformBuffer;
    
    // read byte_buffer from GPU
    let byte_buffer = [1u8; 256 + 8];
    
    let mut buffer = DynamicUniformBuffer::new(&byte_buffer);
    buffer.set_offset(256);
    let vector: test_impl::Vec2i = buffer.create().unwrap();
  12. Use StorageBuffer for runtime-sized arrays

    main

    For data containing arrays whose length is determined at runtime, use the ArrayLength type and the #[shader(size(runtime))] attribute within your ShaderType struct. This works with StorageBuffer to handle the layout of variable-length data.

    use encase::{ShaderType, ArrayLength, StorageBuffer};
    
    #[derive(ShaderType)]
    struct Positions {
        length: ArrayLength,
        #[shader(size(runtime))]
        positions: Vec<test_impl::Vec2f>
    }
    
    let mut positions = Positions {
        length: ArrayLength,
        positions: Vec::from([
            test_impl::Vec2f::from([4.5, 3.4]),
            test_impl::Vec2f::from([1.5, 7.4]),
            test_impl::Vec2f::from([4.3, 1.9]),
        ]),
    };
    
    let mut byte_buffer: Vec<u8> = Vec::new();
    let mut buffer = StorageBuffer::new(&mut byte_buffer);
    buffer.write(&positions).unwrap();
    
    // To read back after GPU modification:
    // byte_buffer[0] = 2; // e.g. changing length on GPU
    // buffer.read(&mut positions).unwrap();
    use encase::{ShaderType, ArrayLength, StorageBuffer};
    
    #[derive(ShaderType)]
    struct Positions {
        length: ArrayLength,
        #[shader(size(runtime))]
        positions: Vec<test_impl::Vec2f>
    }
    
    let mut positions = Positions {
        length: ArrayLength,
        positions: Vec::from([
            test_impl::Vec2f::from([4.5, 3.4]),
            test_impl::Vec2f::from([1.5, 7.4]),
            test_impl::Vec2f::from([4.3, 1.9]),
        ])
    };
    
    let mut byte_buffer: Vec<u8> = Vec::new();
    
    let mut buffer = StorageBuffer::new(&mut byte_buffer);
    buffer.write(&positions).unwrap();
    
    // write byte_buffer to GPU
    
    // change length on GPU side
    byte_buffer[0] = 2;
    
    // read byte_buffer from GPU
    
    let mut buffer = StorageBuffer::new(&mut byte_buffer);
    buffer.read(&mut positions).unwrap();
    
    assert_eq!(positions.positions.len(), 2);