DOCSEARCHbyAlgolia
Submit Docs

HAMi: Heterogeneous AI Computing Virtualization Middleware

website·Indexed 33 days ago

https://project-hami.io

A virtualization middleware for Kubernetes that enables shared GPU resource management across heterogeneous AI accelerators, including NVIDIA, Ascend, Hygon, MetaX, and KunlunXin. It provides features such as vGPU device sharing, topology-aware scheduling, and integration with Volcano, Kueue, and KServe to optimize GPU utilization and memory allocation.

Tokens
492.8K
Snippets
2.2K
Records
3.6K
Agent score
58%

What's inside HAMi

  1. Overview of HAMi support for mthreads GPU sharing

    v1.3.0
    HAMi v1.3.0 supports mthreads.com/vgpu by implementing device-sharing features similar to NVIDIA GPUs. This includes GPU sharing (allocating portions of a GPU to multiple tasks), device memory control, and device core control (specifically tested on MTT S4000 devices) to ensure allocations do not exceed defined boundaries.

  3. Overview of HAMi E2E testing framework

    HAMi utilizes an E2E testing framework designed to validate the entire system workflow and ensure production-level compatibility with the Kubernetes ecosystem. The framework focuses on core functionality, edge cases, error handling, and compatibility across different heterogeneous devices and Kubernetes versions.

  4. Overview of Huawei Ascend NPU Slicing Modes

    The Huawei Ascend device plugin for HAMi supports two resource sharing modes:

    1. Template-based Hard Slicing (vNPU): Memory slicing based on virtualization templates. The system automatically uses the least available template that satisfies the request.
    2. Soft Slicing (hami-vnpu-core): A fine-grained resource sharing mechanism using libvnpu.so interception and limiter token scheduling. This mode is currently restricted to ARM platforms and requires the HAMi scheduler.

  5. Overview of HAMi-DRA

    HAMi-DRA is a Kubernetes mutating webhook that enables Dynamic Resource Allocation (DRA) for GPU workloads. It automatically converts GPU device resource requests into DRA ResourceClaims, providing a user experience consistent with traditional DevicePlugin usage. Key features include automatic resource conversion, resource cleanup, device selection via Pod annotations (UUID, device type), and optional Prometheus metrics monitoring.

  6. Overview of HAMi-core GPU resource controller

    v2.6.0
    HAMi-core is a hook library for CUDA environments that functions as an in-container GPU resource controller. It is utilized by projects such as HAMi and Volcano to manage GPU resources within containers.

  8. Overview of HAMi WebUI Architecture

    v2.9.0

    HAMi WebUI is a visualization and O&M interface for monitoring and managing GPU resources and workloads. It uses a frontend-backend separation architecture:

    • Frontend: A Vue 3 application responsible for pages and user interactions.
    • Backend BFF (Backend For Frontend): A Node/NestJS service responsible for hosting static assets and proxying API requests to the core backend services.
    • Backend Service: A Go-based service that handles the core logic.

  9. Overview of Volcano vGPU for NVIDIA GPUs

    v2.6.0
    Volcano vGPU provides a device-sharing mechanism for NVIDIA devices managed by Volcano, supporting hard isolation of GPU cards via HAMi-core. It is based on the NVIDIA Device Plugin and requires Volcano v1.9 or higher. Note that when using volcano-vgpu, you only need the Volcano vGPU device-plugin; a separate HAMi installation is not required.

  10. Overview of HAMi support for mthreads GPU sharing

    v2.8.0

    HAMi v2.8.0 supports mthreads.com/vgpu by implementing device-sharing features similar to NVIDIA GPUs. Key capabilities include:

    • GPU sharing: Tasks can allocate a portion of a GPU rather than a whole card.
    • Device Memory Control: Allocation of specific device memory sizes (tested on MTT S4000) to ensure boundaries are not exceeded.
    • Device Core Control: Allocation of limited compute cores (tested on MTT S4000) to ensure boundaries are not exceeded.

  11. Overview of HAMi WebUI capabilities

    v2.4.1
    HAMi WebUI is a visual interface for the HAMi ecosystem designed for unified monitoring and analysis of GPU resources and workloads. While HAMi handles the actual GPU resource scheduling and management, the WebUI provides the data presentation and user interaction layer. It allows users to intuitively view cluster running status, resource distribution, and overall system state to reduce the complexity of GPU operations.

  12. Overview of HAMi Vastai device sharing capabilities

    v2.9.0

    HAMi supports sharing vastaitech.com/va (Vastaitech) devices with the following features:

    • Full-Card Mode: Allocates the entire device to a single workload.
    • Die Mode: Partitions the device into sub-units for multiple workloads, with topology awareness to allocate resources on the same AIC when possible.
    • Device UUID Selection: Allows specifying or excluding particular devices via annotations.