Testkube
repository·main·Indexed 11 days ago
https://github.com/kubeshop/testkubeAn open testing platform for Kubernetes that allows teams to run automated tests (API, E2E, Performance, etc.) directly within their cluster infrastructure using existing tools.
What's inside Testkube
- Testkube is an open testing platform designed for Kubernetes environments. It allows engineering teams to define, run, and analyze automated tests using any existing testing tools, scripts, or frameworks (such as API, E2E, Performance, Security, or Infrastructure tests) directly within their Kubernetes infrastructure. It aggregates test results, artifacts, logs, and resource metrics into a centralized location for troubleshooting and reporting.
What is the TestWorkflow Toolkit?
mainThe TestWorkflow Toolkit is a set of helper utilities injected into test containers to coordinate test execution, artifact collection, and service management. It works in tandem with the TestWorkflow Init process.
- Init Process: Handles orchestration, including entrypoints, retry logic, and state management.
- Toolkit: Provides the actual utilities used within tests, such as git operations, artifact management, and parallel execution.
Explore the Testkube Marketplace
mainThe Testkube Marketplace provides an open catalog of ready-to-use Testkube Workflows specifically designed for Infrastructure Testing. You can find these workflows in the testkube-marketplace repository.High Availability (HA) PostgreSQL deployment
mainThe standard Bitnami PostgreSQL chart is intended for single-instance deployments. If you require a High Availability (HA) configuration, you should use the dedicatedpostgresql-hachart instead of the standard chart.Use existing secrets via ExistingSecret
mainThe
ExistingSecretpattern allows users to provide their own existing Kubernetes secrets instead of having the chart generate new ones. This is achieved by mapping expected keys in your templates to specific keys in the user-provided secret.Parameters:
name(string): Name of the existing secret.keyMapping(object): A mapping between the expected key name and the name of the key in the existing secret.
# values.yaml name: mySecret keyMapping: password: myPasswordKey# templates/dpl.yaml (Example usage in a deployment) env: - name: PASSWORD valueFrom: secretKeyRef: name: {{ include "common.secrets.name" (dict "existingSecret" .Values.existingSecret "context" $) }} key: {{ include "common.secrets.key" (dict "existingSecret" .Values.existingSecret "key" "password") }}Use the Testkube expression system for dynamic workflows
mainThe TestWorkflow init process uses an expression engine to handle dynamic configuration, conditional logic, and variable interpolation. Expressions can be used to control when steps run, how they retry, and how they access data from other steps or environment variables.
Step Conditions
Use the
conditionkey to determine if a step should execute. You can reference environment variables or use aliases likealways(which is an alias fortrue).Retry Logic
Control retry behavior using
retry.countandretry.until. Theuntilfield accepts status expressions to define the stopping criteria.Status Expressions
Expressions can evaluate the status of steps using specific syntax:
"passed": Expands tostatus == "passed""failed": Expands tostatus != "passed" && status != "skipped""self.passed": The current step passed."self.failed": The current step failed."step-name.passed": A specific named step passed.
steps: - name: deploy condition: "env.ENVIRONMENT == 'production'" - name: cleanup condition: "always" retry: count: 3 until: "passed"Understand the Testkube CLI structure
mainThe Testkube CLI (
kubectl-testkube, typically invoked astestkube) is akubectlplugin used to manage tests, workflows, and executions. It is designed to work with both the standalone API and the commercial Control Plane APIs.Key architectural layers include:
- Command Groups: Organized by resource type (e.g.,
testworkflows,webhooks,artifacts). - Client Layer: Uses specialized clients for different resources (tests, testworkflows, webhooks) and a dedicated control plane client.
- Configuration: Manages API endpoints, authentication, and multi-environment contexts.
testkube <command> [args]- Command Groups: Organized by resource type (e.g.,
How code changes are applied in development
mainThe development workflow depends on which component you are modifying:
API Server
- With Live Reload: Editing files in
cmd/api-server/,pkg/, orinternal/triggers a local Go compile (~2s) and syncs the binary into the container. The process restarts automatically. - Without Live Reload: Editing files triggers a full Docker rebuild using
build/_local/agent-server.Dockerfile.
kubectl-testkube CLI
- Editing files in
cmd/kubectl-testkube/or shared packages triggers a rebuild of thebuild/_local/kubectl-testkubebinary. - Use the Configure button in the Tilt UI on the
run-cli-commandresource to point the CLI at the local API server (http://localhost:8088) and thetestkube-devnamespace.
Test Workflow Images
- Editing
cmd/testworkflow-init/orcmd/testworkflow-toolkit/triggers a Docker rebuild. New executions will use the updated images.
- With Live Reload: Editing files in
How to trigger and integrate Testkube tests
mainTestkube provides several ways to trigger tests and integrate them into your existing workflows:
Triggering Methods:
- Manually
- On schedules
- From CI/CD or GitOps pipelines
- On Kubernetes Events
- Via the REST API
- Through the Model Context Protocol (MCP)
Integration Methods:
- Webhooks: For event-driven integrations.
- Testkube REST API: For programmatic control and automation.
- MCP Server: For AI-driven troubleshooting and analysis.
Use global variables for credentials in complex deployments
mainIn deployments with multiple dependent sub-charts, passing credentials to every sub-chart individually is difficult. Instead, use
globalvariables to make credentials available to all sub-charts automatically.Example usage:
global.postgresql.auth.username=testuser global.postgresql.auth.password=testpass global.postgresql.auth.database=testdbUnderstand the Testkube local development architecture
mainLocal development runs in a dedicated Kubernetes namespace (default:
testkube-dev). The architecture consists of the following core components:- testkube-api-server: The main API server (HTTP on
:8088, gRPC on:8089) that interacts with databases and NATS. - Databases: Supports PostgreSQL (on
:5432) and/or MongoDB (on:27017). - MinIO: S3-compatible artifact storage (API on
:9000, Web Console on:9090). - NATS: Message queue (on
:4222). - Test Workflow Execution: Dynamically spawned containers including
testworkflow-init(init container) andtestworkflow-toolkit(runtime utilities).
- testkube-api-server: The main API server (HTTP on
Best practices for creating new gRPC endpoints
mainWhen adding new gRPC endpoints to Testkube, follow these architectural guidelines:
- Package Organization: Create an appropriate package, typically under the
testkubedirectory. - Service Separation: Keep gRPC
Servicesseparate. This is critical for the multi-agent architecture (e.g., Runner or Listener agents) to allow for granular splitting ofrpccalls and required permissions. - Protobuf Standards: Adhere to official protobuf best practices:
- Validation: Always run
buf lintto verify structure and compatibility.
- Package Organization: Create an appropriate package, typically under the