LeaderWorkerSet (LWS)

LeaderWorkerSet (LWS) is a Kubernetes API designed to deploy a group of pods as a single unit of replication. It is specifically optimized for AI/ML inference workloads, particularly multi-host inference where a Large Language Model (LLM) is sharded across multiple devices and nodes.

Key characteristics of an LWS group include:

• Unique Pod Identity: Each pod within a group is assigned a unique index from 0 to n-1.
• Parallel Lifecycle: Pods in a group are created in parallel and share the same lifecycle.
• Dual-Template Support: You can specify a template for workers and an optional separate template for the leader pod.
• Group-Level Operations: Scaling, rolling updates, and rollouts are performed at the group level. When a group is updated, all pods within that group are updated together as a single unit.
• Topology-Aware Placement: Supports opt-in co-location of pods within the same group in the same topology.
• All-or-nothing Restart: Supports opt-in policies where all pods in a group are recreated if a single pod fails or a container restarts.

A DisaggregatedSet is a Kubernetes controller and Custom Resource Definition (CRD) designed to support disaggregated inference workloads. It is specifically built for Large Language Model (LLM) inference services where different operational roles (such as prefill, decode, or encode) require different resource profiles or scaling patterns.

When to use DisaggregatedSet

You should use DisaggregatedSet instead of a plain LeaderWorkerSet (LWS) when:

• You are running disaggregated LLM inference (e.g., vLLM with P/D disaggregation, SGLang).
• Different inference phases require different GPU types or different pod group sizes.
• You need to scale prefill and decode replicas independently based on distinct traffic patterns.

When to use plain LeaderWorkerSet (LWS)

Use plain LWS if:

• All inference pods are homogeneous (same model, same resources).
• You do not need to separate prefill from decode.
• You are running training jobs or batch workloads without disaggregation.

The DisaggregatedSet operator is an extension of LeaderWorkerSet designed for workloads that require separating inference phases, such as separating prefill and decode phases onto different infrastructure.

DisaggregatedSet provides:

• Disaggregated Architecture Support: Optimized for workloads where different roles run on separate infrastructure.
• Coordinated N-Dimensional Rollouts: Updates multiple roles (typically 2–10) in lockstep to ensure capacity ratios are preserved during the update process.
• Unified Lifecycle Management: Manages multiple underlying LeaderWorkerSets as a single logical unit.
• Automatic Service Orchestration: Automatically manages headless services for each role to enable discovery and revision-aware routing.
• Advanced Failure Handling: Provides coordinated drain and restart policies across all roles within the disaggregated set.

Starting from v0.9.0, DisaggregatedSet is bundled with the LWS controller manager.

• kubectl/Kustomize: The standard v0.9.0+ manifests include the DisaggregatedSet CRD, controller permissions, and validating webhook automatically.
• Helm: The CRD and permissions are installed by default, but the validating webhook and user-facing ClusterRoles are optional. To enable them, pass --set enableDisaggregatedSet=true during installation.

Upgrading to v0.9.0+ with DisaggregatedSet: If upgrading from a version older than v0.9.0, you must manually apply the DisaggregatedSet CRD before running helm upgrade.

# Enable DisaggregatedSet via Helm
CHART_VERSION=0.9.0
helm install lws oci://registry.k8s.io/lws/charts/lws \
  --version=$CHART_VERSION \
  --namespace lws-system \
  --create-namespace \
  --set enableDisaggregatedSet=true \
  --wait --timeout 300s

# Manual CRD application for upgrades
kubectl apply --server-side \
  -f https://raw.githubusercontent.com/kubernetes-sigs/lws/main/charts/lws/crds/disaggregatedset.x-k8s.io_disaggregatedsets.yaml

helm upgrade lws oci://registry.k8s.io/lws/charts/lws \
  --namespace lws-system \
  --set enableDisaggregatedSet=true

The Pod Controller uses the mapping between the leader's template hash and the ControllerRevision to ensure workers always use the correct specification.

When a pod is reconciled, the controller:

1. Retrieves the templateHash from the pod's labels.
2. Performs a lookup using GetLeaderWorkerSetRevisionFromTemplateHash to find the associated ControllerRevision.
3. Uses that revision to construct the correct WorkerTemplate for the worker StatefulSet.

// Logic used by the Pod Controller to select the correct spec
func Reconcile() {
	controllerRevision := GetLeaderWorkerSetRevisionFromTemplateHash(pod.Labels[templateHash])
	constructWorkerStatefulSetApplyConfiguration(controllerRevision)
}

func constructWorkerStatefulSetApplyConfiguration(currentRevision) {
	currentLws := controllerutils.ApplyRevision(lws, controllerRevision)
	podTemplateSpec := **currentLws.WorkerTemplate.DeepCopy()
}

When a role is configured with scaling.mode: External, the DisaggregatedSet controller automatically creates a DisaggregatedSetRoleScaler resource for that role. This resource exposes the /scale subresource, allowing external autoscalers (HPA, KEDA, etc.) to drive the role's size.

How it works:

1. The external autoscaler writes the desired replica count to DisaggregatedSetRoleScaler.spec.replicas.
2. The DisaggregatedSet controller reads this value and adjusts the underlying LeaderWorkerSet.

DisaggregatedSetRoleScalerSpec:

• spec.replicas: (Required) The desired replica count. The controller seeds this with 1 for new roles or the current LWS replica count during a Static $\rightarrow$ External transition to prevent silent draining to zero.

DisaggregatedSetRoleScalerStatus:

• status.replicas: The observed replica count (aggregated across all current revisions). HPA uses this for ratio math.
• status.selector: A label selector (string) used to match one pod per LWS group (the leader) across all revisions. This ensures HPA's per-pod-metric averaging remains consistent even during rolling updates.