docker-registry-proxy

Since Kind nodes are Docker containers, the proxy must be on the same Docker network.

1. Run the proxy on the kind network with a specific hostname:

docker run --rm --name docker_registry_proxy -it \
       --net kind --hostname docker-registry-proxy \
       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \
       -v $(pwd)/docker_mirror_cache:/docker_mirror_cache \
       -v $(pwd)/docker_mirror_certs:/ca \
       rpardini/docker-registry-proxy:0.7.0

2. Use the following script to configure Kind nodes to use the proxy via containerd:

#!/bin/sh
KIND_NAME=${1-kind}
SETUP_URL=http://docker-registry-proxy:3128/setup/systemd
pids=""
for NODE in $(kind get nodes --name "$KIND_NAME"); do
  docker exec "$NODE" sh -c "\
      curl $SETUP_URL \
      | sed s/docker\.service/containerd\.service/g \
      | sed '/Environment/ s/$/ \"NO_PROXY=127.0.0.0\/8,10.0.0.0\/8,172.16.0.0\/12,192.168.0.0\/16\"/' \
      | bash" & pids="$pids $!" # Configure every node in background
done
wait $pids # Wait for all configurations to end

For in-depth debugging, use the rpardini/docker-registry-proxy:0.7.0-debug image. This version includes nginx-debug and two mitmweb (mitmproxy) instances to inspect traffic at different layers.

Debug Environment Variables:

• DEBUG=true: Enables the mitmweb proxy between the Docker client and the caching layer (accessible on port 8081).
• DEBUG_HUB=true: Enables the mitmweb proxy between the caching layer and DockerHub (accessible on port 8082).
• DEBUG_NGINX=true: Enables nginx-debug and verbose debug logging.

Note: Use these features sparingly; they are not intended for production environments.

docker run --rm --name docker_registry_proxy -it \
       -e DEBUG_NGINX=true -e DEBUG=true -e DEBUG_HUB=true -p 0.0.0.0:8081:8081 -p 0.0.0.0:8082:8082 \
       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \
       -v $(pwd)/docker_mirror_cache:/docker_mirror_cache \
       -v $(pwd)/docker_mirror_certs:/ca \
       rpardini/docker-registry-proxy:0.7.0-debug

To use Docker Desktop for Mac as a client for docker-registry-proxy (running on a different machine), you must inject the CA certificate and the HTTPS_PROXY environment variable directly into the Docker LinuxKit VM.

Warning:

• Do NOT use the Docker Desktop GUI (Preferences > Resources > Proxies) to set manual proxies. Doing so affects all containers, which is undesirable for this setup.
• This is a Man-in-the-Middle (MiTM) configuration and may break during Docker Desktop updates or resets.
• This method does not work if you also require an upstream proxy (e.g., a corporate proxy).

Prerequisites:

1. A factory reset Docker Desktop installation (Troubleshoot > "Reset to Factory defaults").
2. The proxy server must be reachable at a specific IP and port (e.g., 192.168.1.2:3128).

set -e
export DRP_PROXY="192.168.66.100:3129" # Format IP:port, change this 
wget -O - "http://${DRP_PROXY}/" # Make sure you can reach the proxy
# Inject the CA certificate
docker run -it --privileged --pid=host justincormack/nsenter1 \
  /bin/bash -c "wget -O - http://$DRP_PROXY/ca.crt \
 | tee -a /containers/services/docker/lower/etc/ssl/certs/ca-certificates.crt"

# Preserve original config.
docker run -it --privileged --pid=host justincormack/nsenter1 /bin/bash -c "cp /containers/services/docker/config.json /containers/services/docker/config.json.orig"

# Inject the HTTPS_PROXY enviroment variable.
docker run -it --privileged --pid=host justincormack/nsenter1 /bin/bash -c "sed -ibeforedockerproxy -e  's/\"PATH=/\"HTTPS_PROXY=http:\/\/$DRP_PROXY\/\",\"PATH=/ ' /containers/services/docker/config.json"

Standard kops cluster-wide proxy configuration is insufficient because nodeup will fail to download images. Instead, you must use additionalUserData within your InstanceGroup configuration to configure the Docker daemon on each node.

The configuration must perform three main tasks:

1. Configure Docker Proxy: Create a systemd drop-in file at /etc/systemd/system/docker.service.d/http-proxy.conf containing the HTTP_PROXY and HTTPS_PROXY environment variables.
2. Install CA Certificate: Download the proxy's CA certificate from http://docker-registry-proxy.<your_domain>:3128/ca.crt, add it to the system's trusted certificates, and run update-ca-certificates --fresh.
3. Reload and Restart: Run systemctl daemon-reload and systemctl restart docker.service to apply changes.

After applying this configuration to your InstanceGroup, perform a rolling update of your nodes to ensure all subsequent image pulls are cached by the proxy.

apiVersion: kops.k8s.io/v1alpha2
kind: InstanceGroup
metadata:
  labels:
    kops.k8s.io/cluster: spot.k8s.local
  name: spotgroup
spec:
  additionalUserData:
    - name: docker-registry-proxy.sh
      type: text/x-shellscript
      content: |
        #!/bin/sh

        # Add environment vars pointing Docker to use the proxy
        mkdir -p /etc/systemd/system/docker.service.d
        cat << EOD > /etc/systemd/system/docker.service.d/http-proxy.conf
        [Service]
        Environment="HTTP_PROXY=http://docker-registry-proxy.<your_domain>:3128/"
        Environment="HTTPS_PROXY=http://docker-registry-proxy.<your_domain>:3128/"
        EOD

        # Get the CA certificate from the proxy and make it a trusted root.
        curl http://docker-registry-proxy.<your_domain>:3128/ca.crt > /usr/share/ca-certificates/docker_registry_proxy.crt
        echo "docker_registry_proxy.crt" >> /etc/ca-certificates.conf
        update-ca-certificates --fresh

        # Reload systemd
        systemctl daemon-reload

        # Restart dockerd
        systemctl restart docker.service
  image: 099720109477/ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20200528
  machineType: c3.xlarge
  maxSize: 15
  minSize: 2
  mixedInstancesPolicy:
    instances:
    - c3.xlarge
    - c4.xlarge
    - c5.xlarge
    - c5a.xlarge
    onDemandAboveBase: 0
    onDemandBase: 0
    spotAllocationStrategy: capacity-optimized
  nodeLabels:
    kops.k8s.io/instancegroup: spotgroup
  role: Node
  subnets:
  - us-east-1a
  - us-east-1b
  - us-east-1c

To use the docker-registry-proxy on Windows, you must install the proxy's CA certificate into both the Windows certificate store and the WSL2 distribution (if using the WSL2 backend), then configure the proxy settings in Docker Desktop.

1. Obtain the Certificate

Download the certificate from your proxy host using a browser (e.g., http://<proxy-host>:3128/ca.crt) and save it to a local path like d:\ca.crt.

2. Install Certificate in Windows

1. Double-click the .crt file.
2. Select Install Certificate... and click Next.
3. Select Current user and click Next.
4. Select Place all certificates in the following store, click Browse, and choose Trusted Root Certification Authorities.
5. Complete the installation.

Note: If you are NOT using the WSL2 backend, skip the WSL2 steps below and restart Docker Desktop.

3. Install Certificate in WSL2 (Required for WSL2 Backend)

If using the WSL2 backend, you must also import the certificate into the docker-desktop distribution:

1. Identify your distribution (usually docker-desktop) using wsl --list in PowerShell.
2. Enter the distribution shell:

   wsl --distribution docker-desktop

3. Copy the certificate from your Windows drive to the WSL certificate store and update the CA certificates:

   cp /mnt/host/d/ca.crt /usr/local/share/ca-certificates/
   update-ca-certificates

   (Note: You may see a warning about ca-certificates.crt not containing exactly one certificate; the operation has still succeeded.)
4. Type exit to leave the WSL shell.

4. Configure Docker Desktop Proxy Settings

1. Open Docker Desktop Settings.
2. Navigate to Resources > Proxies.
3. Enable the proxy.
4. Set http://<proxy-host>:3128 as both the HTTP and HTTPS URL.

5. Verification

Run the following command in a Windows shell (not inside WSL) to verify connectivity:

docker pull hello-world

If you encounter certificate trust errors, try restarting Docker Desktop or restarting Windows to force a WSL restart.

# Verify the proxy works by pulling an image in a Windows shell
docker pull hello-world

To use the proxy, you must configure your Docker hosts to point to the proxy server and trust its CA certificate.

1. Get the CA certificate: Download it from the proxy via http://<proxy-host>:3128/ca.crt.
2. Configure HTTP/HTTPS Proxy: Set the HTTP_PROXY and HTTPS_PROXY environment variables in the Docker service configuration.
3. Trust the CA: Add the downloaded certificate to your system's trusted root store.
4. Restart Docker: Reload systemd and restart the docker.service.

Warning: If you authenticate to a private registry and pull through the proxy, those images will be served to any client that can reach the proxy, even without authentication. This can make private images public if not managed carefully.

### UBUNTU
# Add environment vars pointing Docker to use the proxy
mkdir -p /etc/systemd/system/docker.service.d
cat << EOD > /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://192.168.66.72:3128/"
Environment="HTTPS_PROXY=http://192.168.66.72:3128/"
EOD

# Get the CA certificate from the proxy and make it a trusted root.
curl http://192.168.66.72:3128/ca.crt > /usr/share/ca-certificates/docker_registry_proxy.crt
echo "docker_registry_proxy.crt" >> /etc/ca-certificates.conf
update-ca-certificates --fresh

# Reload systemd
systemctl daemon-reload

# Restart dockerd
systemctl restart docker.service