kro (Kube Resource Orchestrator)

This example demonstrates a hub-spoke model for managing a fleet of Amazon EKS clusters across multiple regions and accounts. It uses kro, ACK (AWS Controllers for Kubernetes), and Argo CD to provision and bootstrap workload clusters (spokes) via a GitOps flow.

In this architecture, a management cluster (hub) is established first, which contains the necessary EKS capabilities to provision and manage subsequent spoke clusters.

The Dogs vs Cats application consists of three main Kubernetes components:

• Vote App: A Python/Flask frontend application that provides the voting interface. It depends on a Redis endpoint and supports optional Ingress.
• Result App: A Node.js dashboard application that displays voting results. It depends on a PostgreSQL endpoint and a corresponding secret, and supports optional Ingress.
• Worker App: A Java background processor that moves votes from Redis to PostgreSQL. It requires both Redis and PostgreSQL endpoints and is deployed as a Deployment without a Service.

Kube Resource Orchestrator (kro) is a Kubernetes-native tool designed to simplify the creation and management of complex, multi-resource constructs. It allows you to define groupings of Kubernetes resources (both native and custom) as reusable components.

Core Concept: ResourceGraphDefinition

The fundamental custom resource in kro is the ResourceGraphDefinition. It allows you to:

• Define collections of underlying Kubernetes resources.
• Specify dependencies between those resources.
• Include default configurations for their use.

When a ResourceGraphDefinition is applied, the kro controller determines the dependency order and manages the lifecycle of the underlying resources automatically.

KRO uses Kubernetes-style status conditions to communicate the health and state of Resource Graph Definitions (RGD) and Resource Graph Instances.

KRO follows the "single top-level condition" pattern, where a single Ready condition serves as the primary indicator of health. This allows observers to check readiness without needing to understand the internal implementation details or sub-conditions.

KRO prefers normal-true (positive polarity) conditions over the "abnormal-true" style.

Supported condition.Status values:

• True: The condition passes and there are no concerns. All is well.
• False: A fundamental issue requires intervention. The reconciler is stopped from making progress until this is resolved.
• Unknown: The reconciler is making progress but has not yet fulfilled the request.

The Graph Kind (kro.run/v1alpha1) is an atomic primitive for composing Kubernetes resources. It allows you to define a set of nodes that are evaluated in topological order. A Graph acts as a private scope where nodes form a flat namespace, using the Graph's spec as input and providing status as output.

Key characteristics:

• Lifecycle: Creating a Graph converges its resources; deleting it cascades the deletion.
• Isolation: Nodes are private to the Graph and only accessible via the Graph object's own spec and status fields.
• Concurrency: Graphs are reconciled independently and in parallel.
• Patterns: It enables patterns such as static resource bundles (no CRD required), singletons, decorators (reacting to existing resources), and resource aggregation (fan-in).

Graph Revisions provide the foundation for several upcoming orchestration features:

• Revision pinning: Using an annotation or spec field to pin instances to a specific revision number (e.g., new instances target N+1 while existing ones stay on N).
• Leveled topologies: Rolling out revisions across instances in configurable waves (e.g., 10% at a time) with health checks.
• Propagation control integration: Using stable revision identities to control how changes flow through instances.
• Graph diffing: Tools to diff snapshot.spec fields between two revisions.
• Automatic rollback: Falling back to the most recent Active revision if a new revision fails compilation or causes instance failures.
• Revision-aware status: Monitoring rollout progress by surfacing the active revision number in instance status.

kro builds a Directed Acyclic Graph (DAG) to manage resources. Common patterns include:

• Linear Chain: Each resource depends on the previous one (e.g., configmap ──▶ deployment ──▶ service).
• Diamond: Multiple independent resources (e.g., database and cache) converge into a single dependent resource (e.g., app).
• Parallel Branches: Multiple resources reference only the schema and have no interdependencies, allowing kro to create them concurrently to maximize deployment speed.

Graphs do not introduce new privilege boundaries. A Graph author operates at the same privilege level as a standard RGD author (the "infrastructure author" persona). Graph authors have full control over all KRO capabilities within the permissions granted to the KRO controller's service account.

KRO maintains a dual-persona model:

1. Authors (broad permissions) write RGDs and Graphs.
2. Consumers (narrow permissions) interact only with the Kinds that Graphs implement, rather than interacting with Graph objects directly.