repository·main·Indexed 42 days ago
https://github.com/testcontainers/moby-ryukA utility designed to automatically remove Docker containers, networks, volumes, and images based on provided filters after a specified delay, primarily used to clean up resources left behind by automated tests.
You can build Moby Ryuk as a standalone binary or as a Linux Docker container.
To build the binary:
go buildTo build the Linux Docker container with the latest tag:
docker build -f linux/Dockerfile -t testcontainers/ryuk:latest .To start Ryuk using default settings, run the Docker container with access to the Docker socket and the default port mapped.
docker run -v /var/run/docker.sock:/var/run/docker.sock -p 8080:8080 testcontainers/ryuk:latestThe reaper accepts filters over TCP connections. When a client connects, it can send one or more filter strings. Each string is parsed as a URL query string (e.g., label=mykey=myvalue&type=container).
Each filter is processed by addFilter, which converts the query string into filters.Args and stores it. After a filter is successfully added, the reaper sends an ACK\n response back to the client over the connection.
// Example filter string sent over TCP:
label=com.testcontainers.ryuk=true&type=containerorg.testcontainers. To identify a container as being managed by Ryuk, the label org.testcontainers.ryuk is used.The reaper follows a specific lifecycle to ensure safe resource cleanup:
pruner waits for a trigger. Triggers include:context.Context being canceled.ConnectionTimeout.RemoveRetries times if errors occur.If you are running Ryuk locally using go run ., you can simulate connections from a Testcontainer to test label-based filtering using nc (netcat).
To simulate a connection with specific labels:
nc -N localhost 8080 << EOF
label=testing=true&label=testing.sessionid=mysession
label=something
EOFTo send additional session information for monitoring:
printf "label=something_else" | nc -N localhost 8080# Simulate connection with labels
nc -N localhost 8080 << EOF
label=testing=true&label=testing.sessionid=mysession
label=something
EOFYou can customize Ryuk's behavior using the following environment variables. Durations should follow the Go ParseDuration format (e.g., 10s, 1m, 2h).
| Environment Variable | Default | Format | Description |
| ----------------------------- | ------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RYUK_CONNECTION_TIMEOUT` | `60s` | Duration| The duration without receiving any connections which will trigger a shutdown |
| `RYUK_PORT` | `8080` | `uint16`| The port to listen on for connections |
| `RYUK_RECONNECTION_TIMEOUT` | `10s` | Duration| The duration after the last connection closes which will trigger resource clean up and shutdown |
| `RYUK_REQUEST_TIMEOUT` | `10s` | Duration| The timeout for any Docker requests |
| `RYUK_REMOVE_RETRIES` | `10` | `int` | The number of times to retry removing a resource |
| `RYUK_RETRY_OFFSET` | `-1s` | Duration| The offset added to the start time of the prune pass that is used as the minimum resource creation time. Any resource created after this calculated time will trigger a retry to ensure in use resources are not removed |
| `RYUK_CHANGES_RETRY_INTERVAL` | `1s` | Duration| The internal between retries if resource changes (containers, networks, images, and volumes) are detected while pruning |
| `RYUK_VERBOSE` | `false` | `bool` | Whether to enable verbose aka debug logging |
| `RYUK_SHUTDOWN_TIMEOUT` | `10m` | Duration| The duration after shutdown has been requested when the remaining connections are ignored and prune checks start |To create a new reaper instance, use the newReaper function. It accepts a context.Context and a variadic number of reaperOption functions to configure the instance.
By default, if no options are provided:
client.FromEnv).loadConfig().stdout.ctx := context.Background()
// Using default options
r, err := newReaper(ctx)
if err != nil {
log.Fatal(err)
}
// Using custom options
r, err = newReaper(ctx, withConfig(myConfig), withLogger(myLogger), withClient(myDockerClient))The run method starts the reaper's main execution loop. It performs two primary tasks concurrently:
processClients to listen for and handle incoming TCP connections from clients.pruner loop, which waits for prune conditions to be met (e.g., connection timeouts or context cancellation) and then executes the resource removal.The method returns an error if the pruner fails, but will return nil if the error is simply a context.Canceled signal.
err := r.run(ctx)
if err != nil {
// Handle error
}The reaper can be customized during initialization using reaperOption functions. Use these to inject custom dependencies or configurations:
withConfig(cfg config): Sets a custom configuration object. If not used, the reaper loads defaults from the environment.withLogger(logger *slog.Logger): Sets a custom structured logger. Note that using this prevents the reaper from automatically adjusting the log level based on the Verbose configuration setting.withClient(client dockerClient): Sets a custom Docker client implementation.// Example of using specific options
opt := withConfig(config{Verbose: true})
r, err := newReaper(ctx, opt)Ryuk is configured using environment variables. If a variable is not provided, the service uses the default values specified below. These settings control timeouts, retry logic for resource cleanup, and the listening port.
RYUK_CONNECTION_TIMEOUT - Duration without receiving any connections before triggering shutdown (Default: "60s")
RYUK_RECONNECTION_TIMEOUT - Duration after the last connection closes before triggering resource cleanup and shutdown (Default: "10s")
RYUK_REQUEST_TIMEOUT - Timeout for any Docker requests (Default: "10s")
RYUK_REMOVE_RETRIES - Number of times to retry removing a resource (Default: 10)
RYUK_RETRY_OFFSET - Offset added to the start time of the prune pass used as the minimum resource creation time. Resources created after this time trigger a retry to avoid removing in-use resources (Default: "-1s")
RYUK_CHANGES_RETRY_INTERVAL - Interval between retries if resource changes (containers, networks, images, volumes) are detected during pruning (Default: "1s")
RYUK_SHUTDOWN_TIMEOUT - Maximum time the reaper waits for once signalled to shutdown before terminating (Default: "10m")
RYUK_PORT - Port to listen on for connections (Default: 8080)
RYUK_VERBOSE - Enable verbose/debug logging (Default: false)The dockerClient interface defines the subset of Docker API methods required by Ryuk to perform resource reaping. It abstracts the underlying Docker client to allow for easier testing and modularity. Implementations of this interface must provide methods for listing and removing containers, images, networks, and volumes, as well as basic connectivity checks like Ping and NegotiateAPIVersion.
type dockerClient interface {
ContainerList(ctx context.Context, options container.ListOptions) ([]container.Summary, error)
ContainerRemove(ctx context.Context, containerID string, options container.RemoveOptions) error
ImageList(ctx context.Context, options image.ListOptions) ([]image.Summary, error)
ImageRemove(ctx context.Context, imageID string, options image.RemoveOptions) ([]image.DeleteResponse, error)
NetworkList(ctx context.Context, options network.ListOptions) ([]network.Summary, error)
NetworkRemove(ctx context.Context, networkID string) error
VolumeList(ctx context.Context, options volume.ListOptions) (volume.ListResponse, error)
VolumeRemove(ctx context.Context, volumeID string, force bool) error
Ping(ctx context.Context) (types.Ping, error)
NegotiateAPIVersion(ctx context.Context)
}