DOCSEARCHbyAlgolia
Submit Docs

Apache YuniKorn Documentation

website·Indexed 19 days ago

https://yunikorn.apache.org/

Documentation for the Apache YuniKorn scheduler, featuring guides on app-aware scheduling, hierarchical resource queues, and gang scheduling. Includes setup instructions for workloads such as Spark, Flink, Ray, MPI, and TensorFlow, as well as integration with NVIDIA GPUs, Prometheus, and Loki. Covers queue configuration, preemption mechanisms, user/group resolution, and deployment modes for Kubernetes and OpenShift.

Tokens
730.9K
Snippets
2.8K
Records
4.7K
Agent score
75%

What's inside Apache YuniKorn

  1. Overview of YuniKorn Cache Removal Design

    The YuniKorn 1.2.0 design aims to remove the artificial distinction between the cache and scheduler layers. Currently, the Scheduler Core uses a cache for base data tracking and scheduler objects for in-flight details. This dual-layer structure introduces complexity and potential synchronization lags (e.g., YUNIKORN-169) where the scheduler is temporarily 'out of sync' with the cache. The goal is to merge these layers while maintaining existing functionality, ensuring all unit, smoke, and end-to-end tests pass.

  2. Overview of Apache YuniKorn Architecture

    Apache YuniKorn is a light-weight, universal resource scheduler designed for container orchestrator systems. It provides fine-grained resource sharing for multi-tenant, cloud-native environments, supporting mixed workloads including stateless batch workloads and stateful services. While it is commonly deployed as a custom Kubernetes (K8s) scheduler, its architecture is designed to be cross-platform, allowing it to integrate with different ResourceManagers such as Apache Hadoop YARN via a shim layer.

  3. Overview of Historical Usage Tracking in YuniKorn

    YuniKorn 1.6.3 implements an event stream to provide visibility into the application lifecycle, including state changes, asks, and allocation changes. To maintain a stateless architecture and avoid performance issues or out-of-memory failures, YuniKorn does not use an internal datastore for historical data. Instead, it generates an event stream for external tools to consume via a REST interface.

  4. Overview of Historical Usage Tracking in YuniKorn 1.9.0

    YuniKorn 1.9.0 introduces an event stream system to track the lifecycle of applications, nodes, and queues. This allows external tools to build usage patterns without requiring YuniKorn to maintain its own internal datastore, preserving the service's stateless architecture. The system tracks state changes, allocation changes, and usage changes.

  5. Overview of User and Group Resource Usage Tracking

    Apache YuniKorn 1.2.0 implements resource usage tracking at the user and group levels. While resource quotas are traditionally enforced at the queue level, this system allows for limits to be set on specific users or groups. User limit settings take precedence over group limit settings if both are configured. Tracking is performed per queue across the entire queue hierarchy; cluster-level usage is derived from the root queue.

  6. Overview of YuniKorn-K8shim E2E Testing Framework

    YuniKorn-K8shim end-to-end (e2e) tests are built using the Ginkgo and Gomega Behavior-Driven Development (BDD) frameworks. They are designed to ensure consistent behavior of the codebase and catch bugs that unit or integration tests might miss. The test structure is located within the yunikorn-k8shim repository as follows:

    • test/e2e/: Tests for YuniKorn features (Scheduling, Predicates, etc.).
    • test/e2e/framework/configManager: Manages test and cluster configurations.
    • test/e2e/framework/helpers: Utility modules for k8s clients, (de)serializers, and REST API clients.
    • test/e2e/testdata: Test data including configmaps and pod specs.

  7. Overview of YuniKorn Gang Scheduling

    Gang Scheduling ensures that an application is scheduled only when its minimum resource requests can be satisfied. If resources are unavailable, the application waits in a hierarchical queue. This prevents resource fragmentation where partial resources are reserved for an app that cannot yet start, blocking other applications.

  8. Overview of YuniKorn Gang Scheduling

    Gang Scheduling ensures that an application is only scheduled when its minimal resource request can be satisfied. If requirements are not met, the application remains in the queue. This prevents resource segmentation where partial resources are reserved for an app that cannot yet start, which would otherwise block other applications.

  9. Overview of Apache YuniKorn Architecture

    Apache YuniKorn is a light-weight, universal resource scheduler designed for container orchestrator systems. It provides fine-grained resource sharing for mixed workloads (stateless batch workloads and stateful services) in large-scale, multi-tenant, cloud-native environments. While it currently supports Kubernetes as a custom scheduler, its architecture allows for different shim layers to adapt to other ResourceManagers, such as Apache Hadoop YARN.

  10. Overview of YuniKorn Pluggable App Management

    YuniKorn is introducing a pluggable app-management interface to move beyond simple pod-label-based application grouping. This design allows YuniKorn to integrate with 3rd party Kubernetes Operators by retrieving application context and lifecycle state directly from Custom Resource Definitions (CRDs), such as SparkApplication for Spark jobs, rather than relying solely on pod specs.

  11. Overview of YuniKorn Historical Usage Tracking Design

    YuniKorn implements historical usage tracking by generating an event stream for applications, nodes, and queues. This allows external tools to monitor usage patterns without requiring YuniKorn to maintain a stateful datastore, preserving the service's stateless nature and preventing memory-related performance issues.

    Goals:

    • Event streams for Applications (state changes, asks, allocation changes).
    • Event streams for Nodes (state changes, allocation changes).
    • Event streams for Queues (state changes, usage changes).
    • A REST interface for event retrieval.

    Non-Goals:

    • Adding an internal data store for historical data.
    • Providing a built-in UI to display event information.
    • Rebuilding historical data upon recovery.
    • Implementing authentication/authorization on the REST interface.

  12. Overview of Historical Usage Tracking in YuniKorn 1.8.0

    YuniKorn 1.8.0 introduces an event stream system to track the lifecycle and resource usage of applications, nodes, and queues. To maintain a stateless architecture and avoid performance degradation or out-of-memory failures, YuniKorn does not use an internal datastore for historical data. Instead, it generates an event stream that external tools can consume via a REST interface to build usage patterns.

    Goals:

    • Event streams for Applications (state changes, asks, and allocation changes).
    • Event streams for Nodes (state changes, allocation changes).
    • Event streams for Queues (state changes, usage changes).
    • A REST interface for event retrieval.

    Non-Goals:

    • Adding a built-in data store for historical data.
    • Built-in visualization of event information.
    • Rebuilding historical data upon recovery.
    • Authentication/Authorization on the REST interface.