DOCSEARCHbyAlgolia
Submit Docs

Kueue Documentation

repository·main·Indexed 25 days ago

https://github.com/kubernetes-sigs/kueue

A job-level manager for Kubernetes that handles job queueing and admission based on resource availability and priority. Includes documentation for the core manager, the experimental kueue-populator for automating LocalQueue creation, and the experimental kueue-priority-booster for implementing time-sharing fairness mechanisms.

Tokens
194.5K
Snippets
346
Records
935
Agent score
78%

What's inside Kueue

  1. Overview of Kueue features

    main

    Kueue is a job-level manager for Kubernetes that decides when a job should be admitted (pods created) and when it should stop (active pods deleted). Key features include:

    • Job Management: Priority-based queueing using StrictFIFO or BestEffortFIFO strategies.
    • Resource Management: Supports resource flavor fungibility, Fair Sharing, cohorts, and preemption policies.
    • Integrations: Built-in support for BatchJob, Kubeflow training jobs, RayJob, RayCluster, JobSet, and plain Pod/Pod Groups.
    • AdmissionChecks: A mechanism for internal or external components to influence workload admission.
    • Autoscaling: Integration with cluster-autoscaler's provisioningRequest via admissionChecks.
    • Advanced Scheduling: Includes All-or-nothing scheduling (timeout-based), partial admission with reduced parallelism, dynamic reclaim, and Topology-Aware Scheduling (TAS).
    • Multi-cluster: MultiKueue allows searching for capacity and off-loading the main cluster.
    • Observability: Built-in Prometheus metrics and on-demand visibility for pending workloads.

  2. Overview of KEP-77: Dynamically Sized Jobs

    main

    KEP-77 introduces support for horizontal dynamic resizing of Kueue-managed workloads. This allows jobs (such as batch/v1.Job and RayCluster) to scale their parallelism up or down without requiring job suspension, requeueing, or re-acquiring quota.

    Key Capabilities:

    • Horizontal Scaling: Supports increasing or decreasing the number of replicas (pods).
    • In-place Resizing: Adjusts quota usage dynamically without disrupting currently running pods.
    • Workload Slicing: Uses WorkloadSlice objects to manage incremental scale-up requests, providing per-replica control over admission.
    • Scheduling Control: Uses PodSchedulingGates to hold new pods in a gated state until their specific WorkloadSlice is admitted.

    Limitations:

    • No Vertical Scaling: Only horizontal scaling (more/fewer pods) is supported.
    • No Orchestration: Kueue does not initiate the resizing of the parent object (e.g., it doesn't change the parallelism field in the Job spec for you); it reacts to changes made to the parent object.
    • Preemption Behavior: If a resized job must be preempted, Kueue preempts the entire job as a single unit.

  3. Overview of MultiKueue

    main
    MultiKueue is a feature introduced via KEP-693 that provides multi-cluster capabilities to Kueue. It allows users to distribute batch jobs across multiple worker clusters while maintaining specified quota limits. It provides a single entry point (a management cluster) for submitting and monitoring jobs, making multi-cluster job distribution feel like running jobs in a single cluster. It is designed to be compatible with existing Kueue features like priorities, borrowing, and preemptions.

  4. Overview of Topology Aware Scheduling (TAS)

    main

    Topology Aware Scheduling (TAS) is a mechanism in Kueue designed to optimize Job scheduling by leveraging the hierarchical organization of a data center (e.g., nodes within racks, racks within blocks).

    By using node labels to represent this hierarchy, TAS allows users to optimize network throughput for AI/ML workloads that require high pod-to-pod communication. It enables scheduling pods on nodes that are 'close' to each other in the hierarchy to avoid the performance penalties of cross-block or cross-rack communication.

  5. Overview of Concurrent Admission (KEP-8691)

    main

    Concurrent Admission is a feature proposed for Kueue to allow a single Workload to attempt multiple ResourceFlavor (RF) options simultaneously. Instead of selecting one flavor and waiting for admission, Kueue creates clones of the Workload called Variants. Each Variant is scheduled independently on a specific subset of ResourceFlavors.

    Key Benefits

    • Parallel Admission: Users can race multiple long-running AdmissionChecks (e.g., across different accelerator types) to find placement faster.
    • Workload Migration: Enables a path to "upgrade" a Workload from a less preferred flavor (like Spot instances) to a more preferred one (like a Reservation) as it becomes available.
    • Flexible Policies: Supports various migration behaviors, such as migrating only to "higher" RFs, migrating only to a specific subset, or delaying fallback to lower RFs.

    Use Cases

    • Environments where scheduling accuracy (finding the best flavor) is more important than the performance cost of managing multiple variants.
    • Jobs that can tolerate disruption, as migration involves recreating Pods on more preferable Nodes.

  7. Overview of Configurable Resource Transformers (KEP-2937)

    main

    KEP-2937 introduces a mechanism for cluster administrators to customize how a Job's resource requests and limits are translated into the resource requirements of its corresponding Workload.

    This allows Kueue to perform admission and quota calculations using an abstracted or transformed set of resources. Crucially, these transformations only affect how ClusterQueues calculate quota and admission; they do not impact the actual resource requests/limits that the Pods created by the Workload present to the Kubernetes Scheduler.

  8. Overview of Pending-workloads-visibility (KEP-168-2)

    main

    KEP-168-2 introduces a new Extension API server to provide on-demand visibility into pending workloads within Kueue. This API allows users to fetch information about the position of specific workloads and list pending workloads within both ClusterQueue and LocalQueue resources without the scalability and delay limitations of previous implementations.

    Key Capabilities:

    • List pending workloads in a ClusterQueue at specific positions (from X to Y).
    • List pending workloads in a LocalQueue at specific positions (from X to Y).
    • Look up the position of a specific workload in both queue types.

    Non-Goals:

    • Providing Estimated Time of Arrival (ETA).
    • Providing information on workload admissibility.
    • Providing information about requested resources for a workload.

  10. Alternative Resource Transformation Scoping Strategies

    main

    The KEP-2937 proposal discusses several architectural alternatives for applying ResourceTransformation to Kueue objects. While the primary proposal focuses on a monolithic configuration, the following scoping alternatives were evaluated:

    • ClusterQueue Scoped: Extends ClusterQueue with a resourceMappingName (string) or resourceMappingNames ([]string) field. The resource mapping is retrieved from the ClusterQueue.Status instead of a global InfoOption during workload.NewInfo execution.
    • ResourceFlavor Scoped: Involves extending ResourceFlavor with mapping fields. This was deemed significantly more complex due to the dependency between flavor assignment and the effective resource request (which is only known after transformation).
    • Workload Scoped: The finest-grained approach, allowing per-workload custom mapping via annotations. This would likely be applied during workload.AdjustResources.

  11. Understand MultiKueue Architecture and Cluster Roles

    main

    MultiKueue enables multi-cluster job dispatching using a manager-worker architecture.

    Manager Cluster

    Responsible for:

    • Establishing and maintaining connections with worker clusters.
    • Creating and monitoring remote objects (Workloads or Jobs) and keeping local ones in sync.
    • Running the MultiKueue Admission Check Controller to maintain the Active status of AdmissionChecks.

    Quota Tip: The quota for the flavors of a ClusterQueue in the manager cluster should ideally equal the total quota available across all worker clusters to prevent underutilization or excessive dispatching of workloads unlikely to be admitted.

    Worker Cluster

    • Acts as a standalone Kueue cluster.
    • Receives Workloads and Jobs created/deleted by the manager cluster's controller.

    Running Workloads on Manager

    • You can run regular Jobs on the manager cluster using a dedicated ClusterQueue.
    • Limitation: The manager cluster cannot act as a worker for itself.

  12. Understand the Workload concept in Kueue

    main

    A Workload is the unit of admission in Kueue, representing an application that runs to completion (similar to a Kubernetes batch/v1.Job). While Kueue does not directly manipulate Job objects, it manages Workload objects that represent the resource requirements of an arbitrary workload.

    Key characteristics:

    • Admission Unit: Kueue uses Workloads to decide when to admit tasks into queues.
    • Job Integration: For standard Kubernetes Jobs, Kueue automatically creates and syncs a Workload.
    • Concurrent Admission: When enabled on a ClusterQueue, a single Job can be represented by a Parent Workload (aggregating admission status) and multiple Variant Workloads (internal admission attempts constrained to specific ResourceFlavors).