gpu-allocator

repository·main·Indexed 19 days ago

https://github.com/traverse-research/gpu-allocator

A fully Rust-written memory allocator for Vulkan, DirectX 12, and Metal designed to manage GPU memory efficiently across different graphics APIs. Version 0.28.0 supports no_std environments (Rust 1.81+) and provides tools like AllocatorReport for inspecting memory usage, total allocated bytes, and memory block details.

Tokens
12.6K
Snippets
46
Records
53
Agent score
66%

What's inside gpu-allocator

  1. Enable `no_std` support

    main

    To use gpu-allocator in a no_std environment:

    1. Disable default features and enable the hashbrown feature in your Cargo.toml to provide required Hash collections.
    [dependencies]
    gpu-allocator = { version = "0.28.0", default-features = false, features = ["hashbrown"] }

    Note: no_std support requires Rust 1.81 or higher due to dependencies on core::error::Error.

    To support both std and no_std builds in a single workspace, you can define features in your Cargo.toml like this:

    [features]
    default = ["std", "other features"]
    
    std = ["gpu-allocator/std"]
    hashbrown = ["gpu-allocator/hashbrown"]
    other_features = []
    
    [dependencies]
    gpu-allocator = { version = "0.28.0", default-features = false }
    [dependencies]
    gpu-allocator = { version = "0.28.0", default-features = false, features = ["hashbrown", "other features"] }
  2. Inspect memory usage with AllocatorReport

    main

    You can generate an AllocatorReport using Allocator::generate_report() to inspect the current state of memory. This report provides a summary of total allocated bytes versus total capacity, a list of all memory blocks, and details for every live allocation.

    AllocationReport Fields

    • allocations: A Vec<AllocationReport> containing details for all live sub-allocations.
    • blocks: A Vec<MemoryBlockReport> describing the underlying memory blocks.
    • total_allocated_bytes: The sum of memory used by all active allocations.
    • total_capacity_bytes: The total capacity of all memory blocks (including unallocated space).

    AllocationReport Details

    Each AllocationReport includes:

    • name: The string identifier provided during allocation.
    • offset: The byte offset within its memory block.
    • size: The size of the allocation in bytes.

    MemoryBlockReport Details

    Each MemoryBlockReport includes:

    • size: The total size of the block in bytes.
    • allocations: A Range<usize> representing the indices of the associated allocations in the main allocations list.
  3. Use the Global Residency Set for Metal

    main

    If you enabled create_residency_set during allocator initialization, you can retrieve the MTLResidencySet via Allocator::residency_set().

    This set contains all live heaps managed by the allocator. You can attach this set to command buffers or queues to make all allocated resources resident at once.

    Note: You must manually call MTLResidencySet::commit() whenever these resources are used to ensure changes are visible to Metal (e.g., before committing a command buffer).

    if let Some(residency_set) = allocator.residency_set() {
        // Attach residency_set to your command buffer or queue
        // and call commit() before use.
    }
  4. Choose a Vulkan Allocation Scheme

    main

    The AllocationScheme enum determines how the underlying Vulkan memory is handled:

    • DedicatedBuffer(vk::Buffer): Performs a dedicated, driver-managed allocation for a specific buffer. This allows the driver to apply specific optimizations.
    • DedicatedImage(vk::Image): Performs a dedicated, driver-managed allocation for a specific image.
    • GpuAllocatorManaged: The memory is allocated and managed by gpu-allocator (standard sub-allocation behavior).
    #[derive(Copy, Clone, Debug, Eq, PartialEq)]
    pub enum AllocationScheme {
        DedicatedBuffer(vk::Buffer),
        DedicatedImage(vk::Image),
        GpuAllocatorManaged,
    }
  5. Detect memory leaks on shutdown

    main
    The Allocator implements Drop to ensure all remaining memory blocks are destroyed using the provided device. If debug_settings.log_leaks_on_shutdown is enabled, the allocator will automatically call report_memory_leaks and log any unreleased memory to the specified level (e.g., Level::Warn) when the allocator goes out of scope.
  6. Understand AllocationType

    main

    The AllocationType enum categorizes how memory is being used. This is useful for debugging and visualizers to distinguish between different allocation strategies.

    • Free: Unallocated or free memory.
    • Linear: Memory allocated in a linear/sequential fashion.
    • NonLinear: Memory allocated in a non-sequential fashion.
  7. Initialize the D3D12 Allocator

    main

    To use the D3D12 allocator, create an Allocator instance using Allocator::new(). You must provide an AllocatorCreateDesc which includes:

    • device: An ID3D12DeviceVersion (supports ID3D12Device, ID3D12Device10, or ID3D12Device12).
    • debug_settings: An AllocatorDebugSettings object.
    • allocation_sizes: An AllocationSizes object defining block sizes.

    The allocator automatically handles different memory locations (GpuOnly, CpuToGpu, GpuToCpu) and manages heap categories based on the device's resource heap tier.

    // Note: This is a conceptual sketch based on the API
    let desc = AllocatorCreateDesc {
        device: ID3D12DeviceVersion::Device(my_device),
        debug_settings: my_debug_settings,
        allocation_sizes: my_allocation_sizes,
    };
    let mut allocator = Allocator::new(&desc)?;
  8. Initialize the Metal Allocator

    main

    To use the Metal allocator, create an Allocator instance using Allocator::new(). You must provide an AllocatorCreateDesc which requires a MTLDevice.

    Key configuration options in AllocatorCreateDesc:

    • device: The Metal device to allocate from.
    • debug_settings: Configuration for logging and stack traces.
    • allocation_sizes: Settings for memory block sizes.
    • create_residency_set: If set to true, the allocator will manage a MTLResidencySet containing all live heaps. This is only supported on MacOS 15.0+ or iOS 18.0+.
    let desc = AllocatorCreateDesc {
        device: my_mtl_device.clone(),
        debug_settings: AllocatorDebugSettings::default(),
        allocation_sizes: my_allocation_sizes,
        create_residency_set: true,
    };
    let mut allocator = Allocator::new(&desc).expect("Failed to create allocator");
  9. Initialize the Vulkan Allocator

    main

    To use the Vulkan allocator, create an Allocator instance using Allocator::new(). You must provide an AllocatorCreateDesc containing the necessary Vulkan handles and configuration settings.

    Required fields in AllocatorCreateDesc:

    • instance: An ash::Instance.
    • device: An ash::Device.
    • physical_device: A vk::PhysicalDevice.
    • debug_settings: AllocatorDebugSettings for logging and stack traces.
    • buffer_device_address: Boolean indicating if buffer device address support is needed.
    • allocation_sizes: AllocationSizes for managing block sizes.
    let allocator = Allocator::new(&AllocatorCreateDesc {
        instance,
        device,
        physical_device,
        debug_settings,
        buffer_device_address: true,
        allocation_sizes,
    });
  10. Access and copy data to CPU-mapped Allocations

    main

    If an Allocation is host-visible, you can access its memory from the CPU.

    Methods for mapped access:

    • mapped_ptr(): Returns an Option<NonNull<c_void>> to the start of the allocation.
    • mapped_slice(): Returns an Option<&[u8]> representing the entire allocation.
    • mapped_slice_mut(): Returns an Option<&mut [u8]> for mutable access.

    Safe data copying with presser::Slab

    To avoid common pitfalls when copying data (like alignment issues), Allocation implements the presser::Slab trait. You can use try_as_mapped_slab() to get a MappedAllocationSlab, which is a safer wrapper for use with presser helper functions.

    Safety Warning: You must ensure the GPU is not using the memory while you hold a reference to the mapped data. The library cannot statically validate GPU-CPU synchronization.

    // Example: Using try_as_mapped_slab for safe copying
    if let Some(mut slab) = my_allocation.try_as_mapped_slab() {
        // Use presser functions with the slab
        // presser::copy_from_slice_to_offset_with_align(&data, &mut slab, 0, alignment)?;
    }
  11. Allocate memory in D3D12

    main

    To allocate memory for a D3D12 resource:

    1. Define a Direct3D12::D3D12_RESOURCE_DESC.
    2. Create an AllocationCreateDesc using AllocationCreateDesc::from_d3d12_resource_desc, passing the allocator's device, the resource description, a name, and a MemoryLocation (e.g., MemoryLocation::GpuOnly).
    3. Call allocator.allocate(&allocation_desc).
    4. Use device.CreatePlacedResource to create the resource using the allocation's heap() and offset().
    5. Cleanup by dropping the resource and calling allocator.free(allocation).
    use gpu_allocator::d3d12::*;
    use gpu_allocator::MemoryLocation;
    
    
    let buffer_desc = Direct3D12::D3D12_RESOURCE_DESC {
        Dimension: Direct3D12::D3D12_RESOURCE_DIMENSION_BUFFER,
        Alignment: 0,
        Width: 512,
        Height: 1,
        DepthOrArraySize: 1,
        MipLevels: 1,
        Format: Dxgi::Common::DXGI_FORMAT_UNKNOWN,
        SampleDesc: Dxgi::Common::DXGI_SAMPLE_DESC {
            Count: 1,
            Quality: 0,
        },
        Layout: Direct3D12::D3D12_TEXTURE_LAYOUT_ROW_MAJOR,
        Flags: Direct3D12::D3D12_RESOURCE_FLAG_NONE,
    };
    let allocation_desc = AllocationCreateDesc::from_d3d12_resource_desc(
        &allocator.device(),
        &buffer_desc,
        "Example allocation",
        MemoryLocation::GpuOnly,
    );
    let allocation = allocator.allocate(&allocation_desc).unwrap();
    let mut resource: Option<Direct3D12::ID3D12Resource> = None;
    let hr = unsafe {
        device.CreatePlacedResource(
            allocation.heap(),
            allocation.offset(),
            &buffer_desc,
            Direct3D12::D3D12_RESOURCE_STATE_COMMON,
            None,
            &mut resource,
        )
    }?;
    
    // Cleanup
    drop(resource);
    allocator.free(allocation).unwrap();