DOCSEARCHbyAlgolia
Submit Docs

ClusterWareAI Documentation

website·Indexed 35 days ago

https://docs.ice.penguinsolutions.com/clusterware-docs/latest/

Documentation for the ClusterWareAI platform, including an admin guide for cluster management and a comprehensive API reference. It covers primitives for nodes, DPUs, Ethernet switches, InfiniBand networks, image deployment, and software distribution, as well as details on the AI Factory Operations Agent and cluster-wide endpoints.

Tokens
156.2K
Snippets
682
Records
1.2K
Agent score
59%

What's inside ClusterWareAI

  1. Overview of ClusterWareAI Administration Tools

    ClusterWareAI provides tools and methods for maintaining the cluster, controlling cluster usage, batching jobs, managing the job queue, and handling load balancing. Administrative tasks can be performed using either the graphical user interface (GUI) or the command-line interfaces (CLI), as both utilize the same underlying interfaces to the ClusterWareAI database.

  2. Overview of the ClusterWareAI HTTP API

    The ClusterWareAI API is a stateless HTTP-based interface used for cluster management, monitoring, and provisioning. It is primarily used by the official web portal and CLI tools, but can be integrated into custom scripts. Most endpoints are stateless, except for those managing long-running background jobs. Data sent to and received from the API is encoded in JSON format.

  3. Overview of DPU management in ClusterWareAI

    ClusterWareAI supports data processing units (DPUs) associated with compute nodes, specifically NVIDIA BlueField-3 DPUs with DOCA. DPUs are managed using the cw-dpuctl tool. While most compute node actions (like image deployment and PXE booting) apply to DPUs, power control differs: soft power actions (reboot/shutdown) are performed directly on the DPU, while hard power actions (power on/off/cycle) are performed on the associated host compute node.

  8. Overview of cw-modimg image manipulation tool

    cw-modimg (formerly scyld-modimg) is a utility for manipulating node image contents. It allows developers and administrators to create, modify, and manage images used in ClusterWareAI. The tool operates on a local cache and can upload the final version to the ClusterWareAI REST API. If a step fails during execution, changes are discarded and the command returns 1.

  9. Overview of ClusterWareAI Multi-Head Node Configuration

    ClusterWareAI supports active-active configurations where multiple cooperating head nodes share a single replicated database. This allows any head node to provide services for any compute node, including cluster configuration via cw-* tools, booting, power control, and status collection. For full High Availability (HA), a minimum of three cooperating head nodes is required; two head nodes provide limited HA.

  10. Overview of Ignition and Butane for node provisioning

    The ClusterWareAI platform uses Ignition and Butane to partition and format local disks and deploy root file system images.

    • Butane: Translates human-readable YAML configuration files into JSON.
    • Ignition: Consumes the JSON configuration to execute the partitioning and deployment process during the boot sequence.

    This process replaces the traditional kickstart process for locally-installed systems. By default, these systems continue to use the platform for DHCP and status reporting, though this can be reconfigured post-installation.

  11. Overview of Zero-Touch Provisioning (ZTP) for Switches

    The ClusterWareAI platform supports Zero-Touch Provisioning (ZTP) for switches, currently only for Cumulus switches. ZTP allows server-provided scripts to alter the configuration of connected switches. Switches are treated as nodes in the cluster and can be added using the cw-switchctl create command with the switch's MAC address.