DOCSEARCHbyAlgolia
Submit Docs

Koordinator Documentation

website·Indexed 33 days ago

https://koordinator.sh

Documentation for Koordinator, a Kubernetes-based resource orchestration system. It provides advanced scheduling capabilities including Network Topology Aware scheduling, Capacity Scheduling with Elastic Quota, CPU QoS, and GPU/RDMA joint allocation. The platform features components like koord-scheduler, koord-manager, and koord-descheduler to optimize resource utilization and SLO management for large-scale AI training and cloud-native workloads.

Tokens
394.8K
Snippets
1.3K
Records
2.3K
Agent score
60%

What's inside Koordinator

  1. Overview of Koordinator Admission Webhooks

    Koordinator uses Kubernetes admission webhooks to enforce custom policies during the creation, update, and deletion of resources. These are registered via MutatingWebhookConfiguration and ValidatingWebhookConfiguration resources (defined in manifests.yaml). The system ensures proper resource management, quota enforcement, and system stability across various resource types including Pods, Nodes, ConfigMaps, ElasticQuota, and Reservations.

  2. Overview of Network Topology Aware Scheduling

    v1.7
    Network Topology Aware Scheduling in Koordinator (v1.7+) optimizes inter-pod communication for large-scale AI training (e.g., LLMs using Tensor, Pipeline, or Data Parallelism). It ensures pods are scheduled to topology domains with the best performance (lowest latency, highest bandwidth) when resources are available. If resources are insufficient, the scheduler uses job-level preemption based on network topology constraints and records nominations in .status.nominatedNode to maintain consistent placement. This feature integrates with PodGroup/GangGroup semantics.

  3. Overview of PodMigrationJob

    PodMigrationJob is a CRD-based API provided by Koordinator that allows the descheduler or other automatic fault recovery components to evict or delete Pods more safely. It manages the stability risks of the migration process by tracking the status of auditing, resource allocation, and application startup, ensuring that workloads do not fail during migration.

  4. Overview of Koordinator Job-Level Preemption

    Job-level preemption solves the limitation of traditional Kubernetes pod-level preemption, where member pods of a distributed job might not seize resources together. Koordinator ensures that preemption is triggered at the job (GangGroup) level; preemption only occurs if all member pods can be co-scheduled after eviction. Resources are reserved via nominatedNode to maintain scheduling consistency. This feature requires Kubernetes >= 1.18 and Koordinator >= 1.7.0.

  5. Overview of Koordlet DaemonSet

    v1.6
    Koordlet is a DaemonSet deployed on Kubernetes nodes designed to handle co-location resource overcommitment, interference detection, and QoS guarantees. It consists of several independent modules for information collection, data profiling, and QoS management. Some modules, such as the QoS Manager, provide a plugin framework to allow for the easy addition of new resource management strategies.

  6. Overview of ClusterColocationProfile

    v1.7
    The ClusterColocationProfile CRD allows users to define a set of Koordinator-specific configurations (such as QoS class, priority, and resource models) that are automatically injected into newly created Pods via mutating and validating webhooks. This removes the need to modify existing Controllers or Operators to support Koordinator's co-location protocols.

  7. Overview of Koordlet for resource management

    Koordlet is a DaemonSet deployed on Kubernetes nodes designed to handle co-location resource overcommitment, interference detection, and QoS guarantees. It consists of several independent modules for information collection, data profiling, and QoS management. Some modules, such as the QoS Manager, provide a plugin framework to allow for the easy addition of new resource management strategies.

  8. Overview of Koordinator CPU Burst

    v1.7

    CPU Burst is a service level objective (SLO)-aware resource scheduling feature in Koordinator designed to improve the performance of latency-sensitive applications. It prevents application performance degradation caused by kernel CPU throttling.

    • For Linux kernel >= 5.14 (and Anolis OS): It leverages the Burstable CFS Controller to allow containers to accumulate CPU time slices when idle and use them during spikes.
    • For older kernels: The koordlet component detects throttling events and dynamically adjusts the CPU limit to achieve a similar effect.

  9. Overview of Koordinator CPU Suppress mechanism

    v1.7
    The CPU Suppress mechanism, implemented via koordlet on the node, manages the runtime quality of co-located workloads. It dynamically suppresses CPU resources for Best Effort (BE) pods when node load increases and increases their quota when load decreases. This ensures that latency-sensitive (LS, LSR, LSE) pods have sufficient CPU resources. To maintain consistent resource utilization, it is recommended to set the CPU Threshold to the same value as the reserved CPU watermark in the dynamic resource overcommitment model.

  10. Overview of Koordinator CPU Orchestration Workflow

    The fine-grained CPU orchestration process involves three main components:

    1. koordlet: Gathers NUMA topology, CPU topology, and kubelet CPU management policies from the node and updates the NodeResourceTopology CRD. It also hooks CRI requests to apply the scheduled CPU configurations (e.g., cgroup settings) to the runtime.
    2. koord-scheduler: Filters nodes based on NUMA Topology Alignment policies, scores the best node, allocates CPUs during the Reserve phase, and records the allocation in Pod annotations during PreBinding.
    3. koord-runtime-proxy: Works with koordlet to ensure the scheduled CPU results are correctly applied to the container runtime.

  11. Overview of KoordRuntimeProxy

    KoordRuntimeProxy acts as a proxy between kubelet and the container runtime (containerd or dockerd). It intercepts CRI requests to apply resource management policies—such as setting cgroup parameters based on pod priorities for hybrid workloads or applying new isolation policies for specific Linux kernels and CPU architectures—before forwarding the requests to the backend runtime.

  12. Overview of the Enhanced Scheduler Extension

    The Enhanced Scheduler Extension allows developers to extend the Kubernetes scheduling framework to support advanced features (such as Resource Reservation, NUMA Aware Scheduling, and Coscheduling) without modifying the upstream Kubernetes core code. It achieves this by introducing an ExtendedHandle to provide plugins with access to Koordinator-specific resources and by intercepting the plugin initialization process to inject this handle.