Understand the pkg/chaos/lifecycle package
masterThe pkg/chaos/docker package has been renamed to pkg/chaos/lifecycle. This package contains the core lifecycle chaos commands, including:
stopkillrestartpauseremoveexec
repository·master·Indexed 25 days ago
https://github.com/alexei-led/pumbaA chaos testing and network emulation tool for Docker, containerd, and Podman containers. Pumba allows developers to improve system resilience by injecting failures such as container termination (kill, stop, pause, restart), resource stress (CPU, memory, I/O), and network chaos using netem (latency, packet loss, duplication, corruption) and iptables.
The pkg/chaos/docker package has been renamed to pkg/chaos/lifecycle. This package contains the core lifecycle chaos commands, including:
stopkillrestartpauseremoveexecThe Pumba codebase has undergone a modularity review to assess its maintainability for AI agents and developers. The project currently scores 7.4 / 10 on AI-agent maintainability.
Key architectural findings include:
Lister + Netem, Lister + IPTables) which allow for clean mocking.chaos.DockerClient creates implicit coupling.NetemContainer and IPTablesContainer) leak Docker-specific concepts (such as tcimg and pull) into the general contract.pkg/runtime/docker/docker.go is a large monolith (~1.3k LOC) with mixed responsibilities.Pumba provides two complementary tools for network chaos testing by injecting a helper container into the target container's network namespace with NET_ADMIN capabilities:
tc (traffic control). Supports delay, packet loss, corruption, duplication, and rate limiting.iptables. Supports packet loss with random or nth-packet matching.By combining both, you can create realistic asymmetric network conditions.
Pumba supports Podman by interacting with its Docker-compatible API socket using the Docker SDK.
When using the Podman runtime, the podmanClient embeds a Docker client and overrides specific methods to handle Podman-specific behaviors (such as rootless guards, cgroup leaf naming, and sidecar configuration). Most other ctr.Client methods are inherited directly from the Docker delegate and work via the Docker-compat socket.
To test combined network degradation, run pumba netem to limit outgoing bandwidth and pumba iptables to introduce incoming packet loss.
# Limit outgoing bandwidth to 1Mbit/s
pumba netem --tc-image ghcr.io/alexei-led/pumba-alpine-nettools:latest \
--duration 10m rate --rate 1mbit myapp &
# 5% loss on incoming traffic
pumba iptables --iptables-image ghcr.io/alexei-led/pumba-alpine-nettools:latest \
--duration 10m loss --probability 0.05 myapp &Use the pumba stress command to stress-test container resources (CPU, memory, I/O) by running a stress-ng sidecar container. This allows you to simulate resource contention and test how your application handles high load or OOM (Out of Memory) conditions.
Basic Syntax:
pumba stress [options] CONTAINERSKey Options:
--duration, -d (required): How long to run the stress test. Supports unit suffixes like ms, s, m, h.--stressors: Pass specific stress-ng stressors. Note: You must use an = sign when passing values, e.g., --stressors="--cpu 4 --timeout 60s".--stress-image: The Docker image containing stress-ng. Defaults to ghcr.io/alexei-led/stress-ng:latest.--inject-cgroup: Enables same-cgroup injection mode for more realistic resource contention and shared OOM scope (see Same-Cgroup Injection Mode).pumba stress --duration 60s \
--stressors="--cpu 4 --timeout 60s" \
myappPumba provides two main ways to inject network chaos:
Uses Linux tc (traffic control) to add latency, packet loss, duplication, corruption, or rate-limiting.
delay: Add latency (use --time to specify delay in ms).loss: Drop packets (use --probability for percentage).duplicate: Duplicate packets.corrupt: Corrupt packets.rate: Rate-limit packets.Uses iptables to drop packets (ingress and egress).
Tip: If your target container lacks tc or iptables, use the --tc-image flag to spawn a sidecar container that shares the target's network namespace.
# Add 3 seconds network delay to mydb for 5 minutes
pumba netem --duration 5m delay --time 3000 mydb
# Drop 10% of incoming packets to myapp for 2 minutes
pumba iptables --duration 2m loss --probability 0.1 myapp
# Use a sidecar image for containers without nettools
pumba --runtime containerd netem --tc-image ghcr.io/alexei-led/pumba-alpine-nettools:latest \
--duration 5m delay --time 3000 <container-id>To use Podman instead of Docker or Containerd, use the --runtime podman flag. If your Podman socket is in a non-standard location, you can specify it using the --podman-socket flag.
Note on Rootless Mode: Many Pumba features (like netem, iptables, and stress) require rootful Podman. If running in rootless mode, these commands will fail with a specific error message suggesting you use podman machine set --rootful or run as root on Linux.
Whenever an interface signature changes (such as updating Stressor or Lifecycle), you must regenerate the project mocks. Do not hand-edit mock_*.go or mocks/*.go files. Use the following command:
make mocksWhen writing tests with the new request types, use the following mockery matcher style:
mock.AnythingOfType("*container.StressRequest")mock.AnythingOfType("*container.RemoveOpts")Use nodeSelector or nodeAffinity in your DaemonSet specification to target specific node groups (e.g., EKS or GKE pools).
spec:
template:
spec:
# EKS node group
nodeSelector:
alpha.eksctl.io/nodegroup-name: my-node-group
# Or GKE node pool
# nodeSelector:
# cloud.google.com/gke-nodepool: node-poolThe project currently uses a mutable package-level global chaos.DockerClient to manage the runtime client. This creates implicit dependencies and makes testing in isolation difficult.
To improve maintainability and allow for multi-runtime use cases, it is recommended to replace the global with constructor injection.
Recommended Pattern (Option B): Pass a factory closure into the CLI command constructors to make dependencies explicit for both developers and AI agents.
type Runtime func() container.Client
func NewStopCLICommand(ctx context.Context, runtime Runtime) *cli.Command { ... }To ensure the project is in a valid state, use the provided Makefile commands. This is particularly important after making changes to the modularized packages.
make test
make lint
make fmtmake fmt && make lint && make test