OpenKruiseGame (OKG) Documentation

OpenKruiseGame (OKG) is a specialized Kubernetes workload designed for game server scenarios. It addresses limitations in standard Kubernetes workloads (like Deployment and StatefulSet) by providing features essential for gaming, such as hot updates, in-place updates, and granular management of specific game server instances.

Key capabilities include:

• Hot Updates: Support for image-based hot updates and configuration hot reloads to prevent player session interruptions.
• Targeted O&M: Ability to update, delete, or isolate specific game servers (e.g., targeting servers without active players or specific IDs).
• Specialized Networking: Built-in models for fixed IP/port, lossless direct connections, and global acceleration.
• Orchestration & Scaling: Support for complex game server architectures (gateways, engines, policy servers) and auto-scaling.
• Extensibility: Can be used via Kubernetes templates, APIs, or integrated with delivery systems like KubeVela.

OpenKruiseGame (OKG) is a custom Kubernetes workload designed specifically for multi-cloud game server environments. It is a sub-project of the CNCF OpenKruise project.

Unlike standard Kubernetes workloads like Deployment or StatefulSet, OKG provides specialized management features for game servers, including:

• Hot Updates & In-place Upgrades: Update images or configurations without necessarily destroying Pods, preventing player disconnection.
• Targeted Management: Perform directed updates, deletions, or isolations (e.g., targeting specific server IDs or only servers without active players).
• Game-optimized Networking: Supports various models such as fixed IP/ports, lossless direct connection, and global acceleration.
• Complex Orchestration: Facilitates the deployment of multi-component game architectures (e.g., combining gateways, engine servers, and strategy servers).
• Automated Operations: Integrates logging, monitoring, elasticity, and matching services with low-code or zero-code approaches.

OKG can be used via Kubernetes templates/APIs or integrated with delivery systems like KubeVela for visual lifecycle management.

The AmazonWebServices-GlobalAcceleratorCustomRouting plugin is designed for low-latency game servers running as standard Deployment or StatefulSet Pods (where Pod IPs are directly reachable via VPC-CNI).

It uses AWS Global Accelerator Custom Routing accelerators to expose individual Pods. Each accelerator provides two static anycast IPs from two independent network zones for built-in redundancy. The Global Accelerator deterministically maps an anycast IP and a listener port to the Pod IP/port within a registered VPC subnet.

Key characteristics:

• No Data Plane Resources: Unlike the AmazonWebServices-NLB plugin, this plugin does not create any AWS or Kubernetes resources (no NLB, no TargetGroup, no Service).
• Control Plane Only: It manages Allow/Deny status for each Pod on a pre-existing custom routing endpoint group and queries deterministic port mappings.
• Direct Path: There is no load balancer in the data path, and the original client source IP is preserved.
• Status Reporting: The plugin writes the mapping results to the GameServer's network-status field. Game servers should use the Kubernetes Downward API to read this field and report the agaStaticIP:mappedPort to clients.

The AlibabaCloud-SLB plugin uses Alibaba Cloud Classic Layer 4 Load Balancer (SLB/CLB). In this mode, different game servers use different ports on the same SLB instance. The SLB acts as a forwarder without balancing traffic between servers.

Key Characteristics:

• Network Isolation: Supported.
• Health Checks: Highly configurable, supporting both tcp and http protocols.
• Immutability: ExternalTrafficPolicyType and Fixed (if related to IP/Port stability) should not be changed once set.

Core Parameters:

• SlbIds: Comma-separated list of SLB instance IDs.
• PortProtocols: Comma-separated list of port/protocol (e.g., 8000/TCPUDP).
• Fixed: true or false. If true, the mapping remains unchanged even if the Pod is recreated.
• LBHealthCheckType: tcp or http (default: tcp).
• LBHealthCheckInterval: Interval in seconds [1, 50]. Default is 10.

[alibabacloud]
enable = true
[alibabacloud.slb]
# Define the idle port range for SLB port allocation
max_port = 700
min_port = 500

OpenKruiseGame supports multiple update policies, including in-place updates (hot updates), batch updates, and priority-based updates. You can control the scope, sequence, and pace of a rolling update using two key parameters in the GameServerSet (GSS) specification:

1. updatePriority: Set on individual GameServer (GS) resources. Higher values indicate higher priority. When an update is triggered, the GSS will prioritize updating GameServers with higher updatePriority values.
2. partition: Set within the updateStrategy.rollingUpdate configuration of the GameServerSet. This defines the number of game servers that should not be updated during the current update cycle. Setting a non-zero partition allows you to perform canary updates by leaving a specific number of replicas on the old version.

To perform a canary update followed by a full rollout:

1. Increase the updatePriority of the specific GameServer you want to test first.
2. Set the partition value in the GameServerSet to the number of replicas you want to keep on the old version.
3. Once the canary is verified, set partition to 0 to trigger the update for the remaining servers.

# Example GSS updateStrategy configuration
updateStrategy:
  rollingUpdate:
    maxUnavailable: 5
    partition: 2 # Number of servers to keep on the old version
    podUpdatePolicy: InPlaceIfPossible

Logging

• Format: --log-format controls local kubectl logs output. The default is console.
• OTLP Bridge: When --otel-collector-endpoint is configured, the controller uses an OTLP bridge to forward structured logs to the collector. This allows Grafana/Loki to maintain rich metadata even if the local pod logs are in a human-readable console format.
• Trace Correlation: The controller uses logging.FromContextWithTrace(ctx) so that every log emitted within a span automatically includes trace_id and span_id, enabling correlation in Grafana.

Distributed Tracing

• Sampling: Controlled by --otel-sampling-rate (0.0 to 1.0). Invalid values will cause the controller to abort startup.
• Reconciliation Spans: Controllers emit spans for every GameServer and GameServerSet reconciliation. Key attributes include:
  • game.kruise.io.game_server.name / .namespace
  • game.kruise.io.game_server_set.name
  • game.kruise.io.network.status (e.g., ready, not_ready, error, waiting)
• Trace Linking: GameServerManager writes a game.kruise.io/traceparent annotation onto Pods. The admission webhook reads this to link its own spans to the original reconciliation trace.
• Plugin Visibility: Cloud provider plugins (e.g., Alibaba NLB, NodePort) wrap operations in spans to allow inspection of latency and errors per plugin.

The AlibabaCloud-AutoNLBs-V2 plugin follows specific rules to map Kubernetes resources to Alibaba Cloud infrastructure:

1. NLB Allocation

Each Pod is assigned an NLB based on its index: nlbIndex = podIndex / podsPerNLB. Naming Rule: {gssName}-{eipIspType(lowercase)}-{nlbIndex} (e.g., game-server-bgp-0).

2. EIP Allocation

Each NLB corresponds to multiple EIPs (one per zone). Naming Rule: {gssName}-eip-{eipIspType(lowercase)}-{nlbIndex}-z{zoneIndex} (e.g., game-server-eip-bgp-0-z0).

3. Service Mapping

Each Pod corresponds to multiple Services (one per ISP line). Naming Rule: {podName}-{eipIspType(lowercase)} (e.g., game-server-0-bgp).

4. Port Allocation

Pods are allocated unique port ranges on the NLB using the formula: port = minPort + (podIndexInNLB * portCount) + portOffset.

Note: The system automatically skips any ports listed in the BlockPorts configuration.