e2e-framework Documentation

Overview

klient is a Go package designed to abstract the complexities of client-go. While client-go is optimized for building Kubernetes controllers using low-level constructs, klient provides a simplified, programmable interface for interacting with Kubernetes API servers and their resources.

Use Cases

• End-to-End Testing: Quickly performing CRUD (Create, Read, Update, Delete) operations on Kubernetes resources during tests.
• Cluster Control: Managing the underlying compute infrastructure, application deployments, storage, networking, and nodes.
• Simplified API: Interacting with both typed and untyped (unstructured) Kubernetes API objects without the boilerplate of client-go.

Core Categories

• Configuration: Accessing KubeConfig.
• Resource: Managing API resources (creation, search, list, deletion).
• Control: Controlling clusters, applications, storage, network, and nodes.
• Infrastructure: Executing processes locally, remotely, or within pods.

Within a Go test function (e.g., TestCron(t *testing.T)), use the features package to organize test logic into Setup and Assess phases:

1. feature.Setup: Use this to initialize state required for the test, such as setting up a Kubernetes Watcher to listen for specific resource events (e.g., Pod creation).
2. feature.Assess: Use this to perform assertions. An assessment can check if a CRD is installed, create a new resource, or wait for a specific state using the wait package.

Example workflow: Setup a watcher $\rightarrow$ Assess CRD existence $\rightarrow$ Assess resource creation $\rightarrow$ Assess side-effect (e.g., a Pod being created by the controller).

func TestCron(t *testing.T) {
   podCreationSig := make(chan *coreV1.Pod)

   feature := features.New("Cronjob Controller")
   
   // 1. Setup: Watch for events
   feature.Setup(func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
      client := cfg.Client()
      client.Resources(namespace).Watch(&coreV1.PodList{}).WithAddFunc(func(obj interface{}) {
         pod := obj.(*coreV1.Pod)
         if strings.HasPrefix(pod.Name, "cronjob-controller") {
            podCreationSig <- pod
         }
      }).Start(ctx)
      return ctx
   })

   // 2. Assess: Check CRD and create resource
   feature.Assess("Cronjob creation", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
      client := cfg.Client()
      if err := client.Resources().Create(ctx, cronjob); err != nil {
         t.Fatalf("Failed to create cronjob: %s", err)
      }
      // ... wait logic ...
      return ctx
   })
}

The e2e-framework executes tests using a hierarchical lifecycle. You can hook into this lifecycle by registering environment callback functions. These callbacks allow you to inject custom logic (such as logging, resource cleanup, or environment setup) at specific stages of the test execution.

The execution order follows this hierarchy:

1. Setup: Runs once at the very beginning of the environment lifecycle.
2. BeforeEachTest: Runs before each individual Go test function.
3. BeforeEachFeature: Runs before each features.Feature within a test.
4. AfterEachFeature: Runs after each features.Feature within a test.
5. AfterEachTest: Runs after each individual Go test function completes.
6. Finish: Runs once at the very end of the environment lifecycle.

Note that Assess steps within a feature are the actual test assertions and occur between the BeforeEachFeature and AfterEachFeature calls.

// The execution flow for a single test containing features:
// 1. Setup()
// 2. BeforeEachTest()
// 3.   BeforeEachFeature(Feature 1)
// 4.     Assess(Assessment 1)
// 5.   AfterEachFeature(Feature 1)
// 6.   BeforeEachFeature(Feature 2)
// 7.     Assess(Assessment 2)
// 8.   AfterEachFeature(Feature 2)
// 9. AfterEachTest()
// 10. Finish()

Most klient methods use a functional options pattern to allow users to specify optional arguments (like label selectors or retry timeouts) without cluttering the method signature.

Methods typically follow this pattern: func (r *Resources) Method(ctx context.Context, target, ...Option) error

If options are omitted, the framework uses sensible defaults. You can use predefined convenience functions (e.g., resources.WithLabelSelector) or pass custom anonymous functions to modify the options struct directly.

// Example: Using a predefined convenience function for label selectors
if err := res.List(
    context.TODO(), 
    &deps, 
    resources.WithLabelSelector("tier=web"),
); err != nil {
    log.Fatal(err)
}

// Example: Using a custom function to set a RetryTimeout
if err := res.List(
    context.TODO(), 
    &deps,
    func(opts *ListOptions){opts.RetryTimeout=time.Second*30},
); err != nil {
    log.Fatal(err)
}

You can define complex test logic using features.New(). This allows you to encapsulate the lifecycle of a Kubernetes resource:

1. Setup: Create the resource (e.g., a Deployment) using cfg.Client().Resources().Create().
2. Assess: Verify the resource exists and is in the expected state using cfg.Client().Resources().Get(). You can pass data from Assess to Teardown using context.WithValue.
3. Teardown: Clean up the resource using cfg.Client().Resources().Delete().

deploymentFeature := features.New("appsv1/deployment").
	Setup(func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
		// start a deployment
		deployment := newDeployment(cfg.Namespace(), "test-deployment", 1)
		if err := cfg.Client().Resources().Create(ctx, deployment); err != nil {
			t.Fatal(err)
		}
		time.Sleep(2 * time.Second)
		return ctx
	}).
	Assess("deployment creation", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
		var dep appsv1.Deployment
		if err := cfg.Client().Resources().Get(ctx, "test-deployment", cfg.Namespace(), &dep); err != nil {
			t.Fatal(err)
		}
		if &dep != nil {
			t.Logf("deployment found: %s", dep.Name)
		}
		return context.WithValue(ctx, "test-deployment", &dep)
	}).
	Teardown(func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
		dep := ctx.Value("test-deployment").(*appsv1.Deployment)
		if err := cfg.Client().Resources().Delete(ctx, dep); err != nil {
			t.Fatal(err)
		}
		return ctx
	}).Feature()

testenv.Test(t, deploymentFeature)

A HandlerFunc is executed after an object has been decoded and any DecodeOption mutations have been applied. This is primarily used with DecodeEach to perform lifecycle operations on a stream of resources.

type HandlerFunc func(context.Context, k8s.Object) error

Pre-defined Handlers

• CreateHandler(*resources.Resources, ...CreateOption): Creates the decoded objects.
• UpdateHandler(*resources.Resources, ...UpdateOption): Updates the decoded objects.
• DeleteHandler(*resources.Resources, ...DeleteOption): Deletes the decoded objects.
• CreateIfNotExistsHandler(*resources.Resources, ...CreateOption): Creates objects only if they do not already exist.
• IgnoreErrorHandler(HandlerFunc, error): A wrapper that ignores errors returned by the underlying handler.

// Automatically create all resources found in a multi-document YAML
err := DecodeEach(
    context.TODO(), 
    strings.NewReader(multiYaml), 
    CreateHandler(klient.Resources(namespace)),
)