KubeRay Documentation

KubeRay is a Kubernetes operator designed to simplify the deployment and management of Ray applications on Kubernetes. The core of KubeRay consists of three Custom Resource Definitions (CRDs) that manage different workload lifecycles:

• RayCluster: Manages the full lifecycle of a Ray cluster, including creation, deletion, autoscaling, and fault tolerance.
• RayJob: Automates the process of creating a RayCluster and submitting a job. It can be configured to automatically delete the cluster once the job completes.
• RayService: Combines a RayCluster with a Ray Serve deployment graph to provide high availability and zero-downtime upgrades.

Note: For the most up-to-date documentation, refer to the official Ray docs.

The python-apiserver-client is a Python library designed for managing Ray clusters via the KubeRay API server. It provides pythonic interfaces for the full functionality of the API server and the KubeRay operator.

Users can use Python objects to build and manage payloads for:

• Creating, listing, and deleting templates.
• Creating, listing, and deleting Ray clusters.

The client uses structured Python classes to define parameters such as clusters, environment variables, volumes, head groups, worker groups, and job submissions, ensuring type safety and ease of use compared to raw JSON payloads.

KubeRay performance tests are organized by the scale of resources being tested. Each test directory contains a scenario configuration and a results/junit.xml file containing successful Cluster Loader 2 steps from previous runs.

Available test scenarios include:

• 100 RayCluster / 100 RayJob
• 1000 RayCluster / 1000 RayJob
• 5000 RayCluster / 5000 RayJob
• 10000 RayCluster / 10000 RayJob

KubeRay provides two primary API versions for managing Ray clusters on Kubernetes:

• ray.io/v1: The stable API version for production use.
• ray.io/v1alpha1: An experimental API version for testing new features.

Users should generally target ray.io/v1 for stable deployments and resource management.

Beyond the core operator, KubeRay provides several ecosystem components for enhanced management:

• Kubectl Plugin (Beta): Available since v1.3.0, the kubectl ray plugin simplifies common Ray deployment workflows for users, especially those less familiar with Kubernetes.
• KubeRay APIServer (Alpha): Provides a simplified configuration layer for KubeRay resources. It is often used as a backend for custom user interfaces managing KubeRay resources.
• KubeRay Dashboard (Experimental): Introduced in v1.4.0, this dashboard allows users to view and manage KubeRay resources via a UI. It is currently experimental and not yet production-ready.

Supported Features

• View Resources: List Ray jobs and Ray clusters.
• Search & Filter: Use frontend components to find specific resources.
• Management: Delete resources using the select button.
• Test Jobs: Create test jobs (though compute templates are currently limited).

Current Limitations

• Observability Links: Grafana and Logs links do not work because the KubeRay API server does not currently return metrics/logs links.
• Job Creation: Creating jobs via compute templates is not yet fully functional.
• Detail Views: There is no detailed view for individual jobs or clusters.
• Head Node Access: Links to the Ray head node dashboard are not provided due to security/exposure requirements.
• Namespace Selection: The dashboard currently defaults to the default namespace and lacks a namespace selector.

The KubeRay History Server is a service designed to collect, store, and visualize historical logs and metadata from Ray clusters. It provides a web interface for exploring the history of Ray jobs, tasks, actors, and other cluster activities.

It operates using two main components:

1. Collector: A sidecar container that runs within Ray clusters to collect logs and metadata.
2. History Server: A central service that serves a Ray Dashboard-compatible HTTP API and ingests cluster session events on demand.

Supported storage backends include S3/MinIO, Aliyun OSS, and Local Test (for development).

KubeRay APIServer V2 (alpha) is an HTTP proxy server that provides a RESTful interface to the Kubernetes APIServer for managing Ray resources. It mirrors the Kubernetes OpenAPI Spec and KubeRay CRD, making it compatible with existing Kubernetes clients and API interfaces.

There are two ways to use it:

1. As a Go module: Build your own HTTP proxies with custom middleware functions.
2. As a container image: Run the provided community container image to host a proxy server.

Use KubeRay APIServer V2 if you need to manage Ray clusters via HTTP/REST (e.g., from a UI, SDK, or CLI) or if you want to create templates/default values to simplify configuration setup.

KubeRay Pod Pool Virtual Kubelet is an optional, standalone component that provides warmed-up pod pools by registering itself as a virtual kubelet to a Kubernetes cluster.

By using pod pools, KubeRay can request pods that are already active and waiting, effectively skipping the following latency-inducing steps:

• Resource scheduling time: Pods are already allocated.
• Image pulling time: Container images are already present on the nodes.
• Volume preparation time: Volumes are already attached and ready.

This component is designed to accelerate the startup time of Ray clusters within Kubernetes.

RayService enables zero-downtime upgrades for the managed RayCluster. When you modify the spec.rayClusterConfig field in a RayService resource, the following lifecycle occurs:

1. New Cluster Creation: RayService temporarily creates a new RayCluster with the updated configuration.
2. Readiness Wait: It waits for the new RayCluster to become ready.
3. Traffic Switch: Once ready, it updates the selector of the head service managed by RayService (e.g., rayservice-sample-head-svc) to point to the new cluster.
4. Cleanup: The old RayCluster is terminated.

This allows you to update parameters like rayVersion without interrupting service availability.

The KubeRay memory benchmark suite includes three distinct experiment designs to test operator memory scaling:

• Experiment 1 (High CRD churn): Launches a RayCluster (1 head, 0 workers) every 20 seconds until 150 RayCluster custom resources are created.
• Experiment 2 (Pod scaling): Maintains a single RayCluster and adds 5 new worker Pods every 60 seconds until the total reaches 150 Pods.
• Experiment 3 (Cluster scaling): Creates a 5-node RayCluster (1 head + 4 workers) every 60 seconds until 30 RayCluster custom resources are created.