Testkube

repository·main·Indexed 11 days ago

https://github.com/kubeshop/testkube

An open testing platform for Kubernetes that allows teams to run automated tests (API, E2E, Performance, etc.) directly within their cluster infrastructure using existing tools.

Tokens
119K
Snippets
322
Records
437
Agent score
76%

What's inside Testkube

  1. What is Testkube?

    main
    Testkube is an open testing platform designed for Kubernetes environments. It allows engineering teams to define, run, and analyze automated tests using any existing testing tools, scripts, or frameworks (such as API, E2E, Performance, Security, or Infrastructure tests) directly within their Kubernetes infrastructure. It aggregates test results, artifacts, logs, and resource metrics into a centralized location for troubleshooting and reporting.
  2. What is the TestWorkflow Toolkit?

    main

    The TestWorkflow Toolkit is a set of helper utilities injected into test containers to coordinate test execution, artifact collection, and service management. It works in tandem with the TestWorkflow Init process.

    • Init Process: Handles orchestration, including entrypoints, retry logic, and state management.
    • Toolkit: Provides the actual utilities used within tests, such as git operations, artifact management, and parallel execution.
  3. Use existing secrets via ExistingSecret

    main

    The ExistingSecret pattern allows users to provide their own existing Kubernetes secrets instead of having the chart generate new ones. This is achieved by mapping expected keys in your templates to specific keys in the user-provided secret.

    Parameters:

    • name (string): Name of the existing secret.
    • keyMapping (object): A mapping between the expected key name and the name of the key in the existing secret.
    # values.yaml
    name: mySecret
    keyMapping:
      password: myPasswordKey
    # templates/dpl.yaml (Example usage in a deployment)
          env:
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: {{ include "common.secrets.name" (dict "existingSecret" .Values.existingSecret "context" $) }}
                  key: {{ include "common.secrets.key" (dict "existingSecret" .Values.existingSecret "key" "password") }}
  4. Use the Testkube expression system for dynamic workflows

    main

    The TestWorkflow init process uses an expression engine to handle dynamic configuration, conditional logic, and variable interpolation. Expressions can be used to control when steps run, how they retry, and how they access data from other steps or environment variables.

    Step Conditions

    Use the condition key to determine if a step should execute. You can reference environment variables or use aliases like always (which is an alias for true).

    Retry Logic

    Control retry behavior using retry.count and retry.until. The until field accepts status expressions to define the stopping criteria.

    Status Expressions

    Expressions can evaluate the status of steps using specific syntax:

    • "passed": Expands to status == "passed"
    • "failed": Expands to status != "passed" && status != "skipped"
    • "self.passed": The current step passed.
    • "self.failed": The current step failed.
    • "step-name.passed": A specific named step passed.
    steps:
      - name: deploy
        condition: "env.ENVIRONMENT == 'production'"
      - name: cleanup
        condition: "always"
    
    retry:
      count: 3
      until: "passed"
  5. Understand the Testkube CLI structure

    main

    The Testkube CLI (kubectl-testkube, typically invoked as testkube) is a kubectl plugin used to manage tests, workflows, and executions. It is designed to work with both the standalone API and the commercial Control Plane APIs.

    Key architectural layers include:

    • Command Groups: Organized by resource type (e.g., testworkflows, webhooks, artifacts).
    • Client Layer: Uses specialized clients for different resources (tests, testworkflows, webhooks) and a dedicated control plane client.
    • Configuration: Manages API endpoints, authentication, and multi-environment contexts.
    testkube <command> [args]
  6. How code changes are applied in development

    main

    The development workflow depends on which component you are modifying:

    API Server

    • With Live Reload: Editing files in cmd/api-server/, pkg/, or internal/ triggers a local Go compile (~2s) and syncs the binary into the container. The process restarts automatically.
    • Without Live Reload: Editing files triggers a full Docker rebuild using build/_local/agent-server.Dockerfile.

    kubectl-testkube CLI

    • Editing files in cmd/kubectl-testkube/ or shared packages triggers a rebuild of the build/_local/kubectl-testkube binary.
    • Use the Configure button in the Tilt UI on the run-cli-command resource to point the CLI at the local API server (http://localhost:8088) and the testkube-dev namespace.

    Test Workflow Images

    • Editing cmd/testworkflow-init/ or cmd/testworkflow-toolkit/ triggers a Docker rebuild. New executions will use the updated images.
  7. How to trigger and integrate Testkube tests

    main

    Testkube provides several ways to trigger tests and integrate them into your existing workflows:

    • Triggering Methods:

      • Manually
      • On schedules
      • From CI/CD or GitOps pipelines
      • On Kubernetes Events
      • Via the REST API
      • Through the Model Context Protocol (MCP)
    • Integration Methods:

      • Webhooks: For event-driven integrations.
      • Testkube REST API: For programmatic control and automation.
      • MCP Server: For AI-driven troubleshooting and analysis.
  8. Use global variables for credentials in complex deployments

    main

    In deployments with multiple dependent sub-charts, passing credentials to every sub-chart individually is difficult. Instead, use global variables to make credentials available to all sub-charts automatically.

    Example usage:

    global.postgresql.auth.username=testuser
    global.postgresql.auth.password=testpass
    global.postgresql.auth.database=testdb
  9. Understand the Testkube local development architecture

    main

    Local development runs in a dedicated Kubernetes namespace (default: testkube-dev). The architecture consists of the following core components:

    • testkube-api-server: The main API server (HTTP on :8088, gRPC on :8089) that interacts with databases and NATS.
    • Databases: Supports PostgreSQL (on :5432) and/or MongoDB (on :27017).
    • MinIO: S3-compatible artifact storage (API on :9000, Web Console on :9090).
    • NATS: Message queue (on :4222).
    • Test Workflow Execution: Dynamically spawned containers including testworkflow-init (init container) and testworkflow-toolkit (runtime utilities).
  10. Best practices for creating new gRPC endpoints

    main

    When adding new gRPC endpoints to Testkube, follow these architectural guidelines:

    1. Package Organization: Create an appropriate package, typically under the testkube directory.
    2. Service Separation: Keep gRPC Services separate. This is critical for the multi-agent architecture (e.g., Runner or Listener agents) to allow for granular splitting of rpc calls and required permissions.
    3. Protobuf Standards: Adhere to official protobuf best practices:
    4. Validation: Always run buf lint to verify structure and compatibility.