spotify-docker-client

repository·master·Indexed 23 days ago

https://github.com/spotify/docker-client

A Java client for interacting with the Docker Engine API, used for programmatically managing Docker containers, images, networks, volumes, and other Docker resources. It supports connection via Unix sockets or HTTPS, provides authentication for private registries, and includes a builder for configuring timeouts and connection pooling.

Tokens
5.5K
Snippets
15
Records
25
Agent score
79%

What's inside spotify-docker-client

  1. Configure Connection Pooling

    master

    The client uses Apache HTTP client with a shared connection pool. The default pool size is 100 concurrent requests. If you need to handle more concurrent operations (e.g., waiting on many containers via DockerClient.waitContainer), increase the pool size using .connectionPoolSize().

    Note: The connect timeout applies to acquiring a connection from the pool. If the pool is exhausted, a DockerTimeoutException may be thrown.

    final DockerClient docker = DefaultDockerClient.fromEnv()
        .connectionPoolSize(SOME_LARGE_NUMBER)
        .build();
  2. Mount host directories in a container

    master

    To mount host directories (bind mounts) into a container, configure a HostConfig and pass it to the ContainerConfig.

    Bind Methods

    1. String format: Pass strings to binds() in the format "local_path:container_path" (for read/write) or "local_path:container_path:ro" (for read-only).
    2. Bind object: Use Bind.from(source).to(destination).
      • source can be a String (local path) or a Volume object.
      • destination must be a String (path inside the container).
      • Use .readOnly(true) for read-only mounts.

    Implementation Pattern

    Use appendBinds() or appendBinds(Bind) on the HostConfig.Builder to add multiple mounts.

    Note on API Versions: For Docker API version 1.20+ (Docker 1.8.x+), container volume information is returned under the key "Mounts". Use ContainerInfo.mounts() instead of the deprecated ContainerInfo.volumes().

    final HostConfig hostConfig = HostConfig.builder()
      .appendBinds("/local/path:/remote/path") // String format
      .appendBinds(Bind.from("/another/local/path")
                   .to("/another/remote/path")
                   .readOnly(true)
                   .build()) // Bind object format
      .build();
    
    final ContainerConfig config = ContainerConfig.builder()
      .image("busybox:latest")
      .hostConfig(hostConfig)
      .build();
  3. Authenticate to private registries

    master

    Authentication for building, pushing, or pulling images is handled via the RegistryAuthSupplier interface.

    Available implementations:

    • auth.ConfigFileRegistryAuthSupplier: Reads from ~/.dockercfg or ~/.docker/config.json.
    • auth.FixedRegistryAuthSupplier: Uses fixed RegistryAuth and RegistryConfigs POJOs.
    • auth.gcr.ContainerRegistryAuthSupplier: Fetches tokens for Google Container Registry.
    • auth.MultiRegistryAuthSupplier: Combines multiple implementations.

    Since version 8.7.0, ConfigFileRegistryAuthSupplier is enabled by default unless you explicitly provide other authentication via .dockerAuth(), .registryAuth(), or .registryAuthSupplier() in the builder.

  4. Execute commands inside a container

    master

    The exec API allows you to run processes inside an existing container.

    1. Create: Use execCreate(containerId, commandArray) to define the execution. It returns an ExecCreate object containing an id.
    2. Start/Run: Use execStart(execId) to run the command. This returns a LogStream which you can read to capture output.
    3. Inspect: Use execInspect(execId) to check the ExecState, including exitCode(), running(), and whether openStdout(), openStderr(), or openStdin() are active.
    4. Resize: Use execResizeTty(execId, height, width) to resize the TTY of a running exec instance.
  5. Create a DockerClient

    master

    You can instantiate a DockerClient using environment variables or a builder. DefaultDockerClient.fromEnv() uses DOCKER_HOST and DOCKER_CERT_PATH to configure the client. DefaultDockerClient.builder() allows for manual configuration of timeouts, connection pools, and other parameters.

    // Create a client based on DOCKER_HOST and DOCKER_CERT_PATH env vars
    final DockerClient docker = DefaultDockerClient.fromEnv().build();
    
    // or use the builder
    final DockerClient docker = DefaultDockerClient.builder()
      // Set various options
      .build();
  6. Manage Docker volumes

    master

    Use the volume API to manage persistent storage.

    • List volumes: listVolumes() returns a VolumeList containing a list of Volume objects and any warnings.
    • Create a volume:
      • Named volume: Use Volume.builder() to specify name, driver, and labels, then call createVolume(volume).
      • Anonymous volume: Call createVolume() with no arguments.
    • Inspect: inspectVolume(volumeName) returns metadata for the volume.
    • Remove: removeVolume(volumeName) or removeVolume(volumeObject) deletes the volume.
    // Create a named volume with labels
    final Volume toCreate = Volume.builder()
      .name("volumeName")
      .driver("local")
      .labels(ImmutableMap.of("foo", "bar"))
      .build();
    final Volume created = docker.createVolume(toCreate);
    
    // Remove a volume
    docker.removeVolume("volumeName");
  7. Save and load images as tarballs

    master

    You can export images as tar streams using save or saveMultiple and import them using load.

    • Save a single image: save(imageName) returns an InputStream of the tarball.
    • Save multiple images: saveMultiple(name1, name2, ...) returns an InputStream containing multiple images.
    • Load images: load(InputStream) accepts an InputStream containing one or more images/tags in tar format.
  8. Configure DOCKER_HOST for Docker for Mac

    master

    When using Docker for Mac with DefaultDockerClient.fromEnv(), you may need to explicitly set the DOCKER_HOST environment variable if you are on an older version of the client.

    For Docker for Mac, use: DOCKER_HOST=unix:///var/run/docker.sock

    Note: As of docker-client version 4.0.8, DefaultDockerClient.fromEnv() uses unix:///var/run/docker.sock on OS X by default.

  9. Install docker-client via Maven

    master

    To use docker-client in your Java project, add the following dependency to your pom.xml.

    Note: If your project uses Jersey 1.x, you should use the shaded classifier to avoid dependency conflicts with Jersey 2.x.

    <!-- Standard version -->
    <dependency>
      <groupId>com.spotify</groupId>
      <artifactId>docker-client</artifactId>
      <version>LATEST-VERSION</version>
    </dependency>
    
    <!-- Shaded version (use this if you have Jersey 1.x conflicts) -->
    <dependency>
      <groupId>com.spotify</groupId>
      <artifactId>docker-client</artifactId>
      <classifier>shaded</classifier>
      <version>LATEST-VERSION</version>
    </dependency>
  10. Manage Docker images

    master

    Use the docker client to perform lifecycle operations on images, including listing, building, pulling, pushing, tagging, and removing them.

    • List images: Use listImages with ListImagesParam to filter by labels.
    • Build images: Use build with a path to a Dockerfile and a ProgressHandler to capture the resulting image ID.
    • Create images: Create images by pulling from a registry (using RegistryAuth) or by createing from an InputStream (loading a tarball).
    • Inspect/History: Use inspectImage for metadata and history for the image's layer history.
    • Tagging: Use tag to assign names. The method supports a force boolean to re-assign tags.
    • Push/Remove: Use push to upload to a registry and removeImage to delete locally.
    // List images with labels
    final List<Image> quxImages = docker.listImages(ListImagesParam.withLabel("foo", "qux"));
    
    // Build an image
    final String returnedImageId = docker.build(
        Paths.get(dockerDirectory), "test", new ProgressHandler() {
          @Override
          public void progress(ProgressMessage message) throws DockerException {
            final String imageId = message.buildImageId();
            // handle imageId
          }
        });
    
    // Pull an image with authentication
    final RegistryAuth registryAuth = RegistryAuth.builder()
      .email(AUTH_EMAIL)
      .username(AUTH_USERNAME)
      .password(AUTH_PASSWORD)
      .build();
    docker.pull("dxia2/scratch-private:latest", registryAuth);
    
    // Tag an image
    docker.tag("busybox:latest", "testRepo/tagForce:sometag");
    
    // Force-re-assign tag
    docker.tag("busybox:buildroot-2014.02", "testRepo/tagForce:sometag", true);
  11. Troubleshoot HTTP 500 errors

    master

    If the Docker daemon encounters an unexpected error, the client will throw an exception such as: com.spotify.docker.client.shaded.javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error.

    Resolution: This is a server-side error from the Docker daemon. Check the Docker daemon logs on the host machine (typically at /var/log/docker.log or /var/log/upstart/docker.log) to find the root cause.