DOCSEARCHbyAlgolia
Submit Docs

OpenKruise Agents

repository·master·Indexed 42 days ago

https://github.com/openkruise/agents

Specialized management for AI agent workloads on Kubernetes, providing sandboxed environments with features such as hibernation, resource pooling, and GPU memory checkpointing. Includes components like sandbox-gateway for Envoy-based routing and sandbox-manager for lifecycle operations, with support for E2B Python SDK integration and SandboxSet resources.

Tokens
113.7K
Snippets
213
Records
457
Agent score
54%

What's inside OpenKruise Agents

  1. Overview of Sandbox Prometheus Metrics Observability

    master

    OpenKruise Agents provides comprehensive Prometheus metrics observability for its sandbox ecosystem, following kube-state-metrics design patterns. Metrics are exposed via the standard /metrics HTTP endpoint and are designed for Prometheus scraping and Grafana visualization.

    The observability coverage spans four primary components:

    • Sandbox Controller: Tracks sandbox instance lifecycle, phases, and conditions.
    • SandboxSet Controller: Tracks resource pool replica metrics.
    • SandboxClaim Controller: Tracks operation phases and timing metrics.
    • Sandbox Manager: Tracks API operation latency, success/failure counters, claim/clone stage-level metrics, and route sync metrics.
    • Proxy: Provides route table and peer topology metrics.
    • E2B Server: Provides snapshot operation metrics.

  2. Overview of OpenKruise Agents

    master

    OpenKruise Agents is a sub-project of OpenKruise (a CNCF workload project) designed to manage AI agent workloads in Kubernetes. It provides vendor-neutral sandboxes optimized for:

    1. Isolated environments: For diverse tool usage by AI agents.
    2. Persistent cloud environments: Network-accessible workspaces for research notebooks and development.
    3. Reinforcement learning: Supporting human-in-the-loop and open-world training.
    4. Big data training: Focused on rapid startup times and robust fault tolerance.

    Key capabilities include resource pooling/dynamic resizing, sandbox hibernation (including memory, R/W layers, and GPU memory), and session management with efficient traffic routing.

  3. Overview of the okactl CLI tool

    master

    okactl is a kubectl-style command-line interface designed for OpenKruise Agents. It provides simplified, user-friendly commands for managing sandbox resources, wrapping complex Kubernetes API operations (like JSON patches or specific CRD field paths) into memorable commands. It is intended for operators who need to perform routine tasks like scaling, image updates, or container restarts without needing deep expertise in Kubernetes internals.

    Key features include:

    • Input Validation: Verifies container names and template modes before submission to prevent silent failures.
    • kubectl Conventions: Supports standard flags like -n for namespace and --kubeconfig.
    • Resource Short Names: Supports sbs (sandboxset), sbx (sandbox), and suo (sandboxupdateops).
    • Stateless Design: Operates via direct API calls without requiring a local daemon or database.

  4. Understand OpenTelemetry-based Distributed Tracing for Sandbox Lifecycle

    master

    The OpenKruise Agents project implements distributed tracing for sandbox lifecycle management using the OpenTelemetry (OTel) SDK. This tracing architecture provides visibility into two primary components:

    1. sandbox-manager: Provides API-level tracing for all E2B-compatible HTTP endpoints. Because sandbox-manager requests are synchronous (e.g., CreateSandbox waits for the Pod to be Ready), the root Span's duration represents the full end-to-end latency of the user request.
    2. sandbox-controller: Provides tracing for phase transitions, Pod operations, Checkpoint, and Reuse flows within the Reconcile loop. Since a single user operation can trigger multiple Reconcile passes, each pass produces a sibling Span.

    Key Concept: Trace Correlation Trace context is propagated from the sandbox-manager to the sandbox-controller using the agents.kruise.io/trace-context Sandbox CRD Annotation. This allows end-to-end correlation of a single user operation (like create, delete, pause, or resume) across both components. Each unique user operation is treated as an independent Trace with its own unique TraceID.

  5. Overview of SandboxSet Autoscaler

    master

    The SandboxSet autoscaler provides intelligent and dynamic management of pre-warmed sandbox resource pools. While standard SandboxSet uses fixed-size pools, the autoscaler allows for two types of scaling policies to optimize for latency-sensitive agent workloads:

    1. Cron-based policies: Enables time-driven scaling based on recurring schedules (e.g., scaling up during business hours and down during off-peak periods).
    2. Capacity-based policies: Enables demand-driven scaling based on available resource watermarks, ensuring sufficient idle capacity is maintained to prevent startup latency spikes while preventing over-provisioning.

    This is particularly useful for agents that require near-real-time startup and stable latency, especially during batch startup or fan-out execution scenarios.

  7. Configure Sandbox Auto-Pause and Resume

    master

    The Sandbox CRD supports automated lifecycle management through two new fields: Spec.Probes and AutoPausePolicy. This allows you to reclaim compute resources during idle periods by automatically pausing sandboxes and resuming them based on activity detection.

    Core Mechanisms

    1. Probing (Spec.Probes): You define generic, named probes. The controller executes these periodically and writes the results to SandboxStatus.Conditions. Probes are agnostic of their purpose; they simply report state.
    2. Decisions (AutoPausePolicy): This field defines the strategy for pausing and resuming. It references the names of the probes defined in Spec.Probes.

    Operational Modes

    • Mode 1: Probe-Driven Decisions: When AutoPausePolicy is configured, the controller uses probe results to automatically toggle the Spec.Paused field.
    • Mode 2: Probe-Only Reporting: If AutoPausePolicy is not configured, the controller acts in a passive mode. It will still execute probes and report their results to SandboxStatus.Conditions, but it will not make any automatic pause/resume decisions. This allows external platforms to implement their own custom logic based on the reported conditions.
    # Conceptual representation of the new Sandbox fields
apiVersion: agents.openkruise.io/v1alpha1
kind: Sandbox
metadata:
  name: my-agent-sandbox
spec:
  probes:
    - name: activity-detector
      # probe configuration details...
  autoPausePolicy:
    pause:
      probeName: activity-detector
      # conditions for pausing...
    resume:
      probeName: activity-detector
      # conditions for resuming...

  8. Calculate Sandbox Phase Durations via Timestamps

    master

    You can analyze how long a Sandbox stays in various lifecycle phases by performing timestamp subtraction on the exported metrics. This method does not require additional metrics but relies on the chain of transition timestamps.

    Timestamp Chain: sandbox_created $\rightarrow$ sandbox_status_ready_time $\rightarrow$ sandbox_status_paused_time $\rightarrow$ sandbox_status_resumed_time $\rightarrow$ sandbox_deletion_timestamp.

    Important Notes:

    • Timestamp metrics only record the most recent transition value. In multiple pause/resume cycles, earlier timestamps are overwritten.
    • To isolate the latency of the Resuming operation itself (excluding the time spent waiting in the Paused state), use the sandbox_resume_duration_seconds Histogram instead of timestamp subtraction.
    # Pending phase duration (from creation to ready)
sandbox_status_ready_time - sandbox_created

# Running phase duration (from ready to paused, first run cycle)
sandbox_status_paused_time - sandbox_status_ready_time

# Paused phase duration (from pause completion to resume completion)
sandbox_status_resumed_time - sandbox_status_paused_time

# Full lifecycle duration (from creation to deletion)
sandbox_deletion_timestamp - sandbox_created

  9. Understand OTel Non-Blocking Async Export

    master

    Tracing in OpenKruise Agents is designed to have minimal impact on business performance through an asynchronous, non-blocking export mechanism:

    1. Async Batching: The system uses a BatchSpanProcessor. When span.End() is called, the data is simply enqueued in memory, and the function returns immediately.
    2. Background Export: A background goroutine handles the periodic batching and transmission of spans to the OTLP gRPC exporter.
    3. Graceful Degradation: If the OTel Collector is unavailable, spans accumulate in the in-memory queue and are eventually dropped. This ensures that tracing issues do not block or crash the core business logic.

  10. Understand Sandbox Reuse and Scale-Down Priority

    master

    When using the SandboxSet controller, sandboxes can be reused to improve efficiency. A reused sandbox is treated as an Available sandbox once it returns to the pool.

    If the SandboxSet needs to scale down (when spec.replicas < status.replicas), it follows a specific priority order to decide which sandboxes to delete first (highest priority deleted first):

    1. Creating sandboxes: Newest first (lowest discard cost).
    2. Reused sandboxes: Those with reuseCount > 0. Sandboxes with a higher reuseCount are deleted first to mitigate the risk of accumulated state leakage.
    3. Running but not Ready sandboxes: Cannot be claimed.
    4. Available sandboxes: Newest first (highest discard cost).

  11. Understand the Multi-Reconcile Sibling Span model

    master

    The Sandbox lifecycle involves long-running operations where the sandbox-manager waits for a result, but the sandbox-controller performs multiple asynchronous Reconcile passes to reach that state.

    To model this, the system uses a Sibling Span Model:

    • The sandbox-manager operation (e.g., CreateSandbox) holds a long-lived Root Span representing the total user-perceived latency.
    • Each individual Reconcile pass in the controller creates its own Sibling Span (sharing the same TraceID and ParentSpanID as the root). These spans are short-lived and end immediately after the reconcile loop completes.

    Why this matters: This model makes bottlenecks visible. If the Root Span lasts 30 seconds but the combined duration of all controller Reconcile spans is only 280ms, the gap (e.g., 29.7s) clearly indicates that the delay is caused by external factors like Pod startup (image pulling, container initialization) rather than controller logic.

  12. Understand Quota Enforcement Behavior (Fail-Open)

    master

    The system is designed to prioritize availability over strict enforcement during infrastructure issues.

    Fail-Open Policy: If there is a transport error or the Redis circuit breaker is open, the system will fail-open. This means quota enforcement is bypassed, and sandboxes will be allowed to be created even if they technically exceed the configured limits. This state is temporary and is intended to be self-healed once the backend becomes healthy again.