DOCSEARCHbyAlgolia
Submit Docs

offen/docker-volume-backup

repository·main·Indexed 25 days ago

https://github.com/offen/docker-volume-backup

A lightweight companion container for performing recurring or one-off backups of Docker volumes. It supports local storage and remote providers including S3, WebDAV, Azure Blob Storage, Dropbox, Google Drive, and SSH. Key features include GPG encryption, backup rotation via BACKUP_RETENTION_DAYS, and the ability to stop containers during backup using the docker-volume-backup.stop-during-backup label.

Tokens
18.4K
Snippets
51
Records
84
Agent score
86%

What's inside docker-volume-backup

  1. Handle file uploads using third party tools via lifecycle hooks

    main

    If you need to use an unsupported storage backend or a third-party tool (like rsync or rclone) for uploading backups, you can extend the base image with the required binaries and use the docker-volume-backup.copy-post label to trigger a command after the backup archive is created.

    When the command is invoked, the filepath of the generated tar archive is passed to your command via the environment variable COMMAND_RUNTIME_BACKUP_FILEPATH (or $$COMMAND_RUNTIME_BACKUP_FILEPATH in Docker Compose files).

    # 1. Build a custom image with the required tool
FROM offen/docker-volume-backup:v2

RUN apk add rsync
    # 2. Use the custom image and define the copy-post label
services:
  backup:
    image: your-custom-image
    restart: always
    environment:
      BACKUP_FILENAME: "daily-backup-%Y-%m-%dT%H-%M-%S.tar.gz"
      BACKUP_CRON_EXPRESSION: "0 2 * * *"
    labels:
      # Use the lifecycle hook to run your tool
      # Note: Use $$ to escape the $ in Docker Compose
      - docker-volume-backup.copy-post=/bin/sh -c 'rsync $$COMMAND_RUNTIME_BACKUP_FILEPATH /destination'
    volumes:
      - app_data:/backup/app_data:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro

volumes:
  app_data:

  2. Run multiple backup schedules in the same container

    main

    You can run multiple backup schedules with different configurations within a single container by mounting multiple .env formatted configuration files into the /etc/dockervolumebackup/conf.d directory.

    Each configuration file results in a separate cronjob. If configuration values exist in both the global environment and a specific config file, the config file takes precedence.

    Important constraints:

    • Serial Execution: The backup command requires an exclusive lock. If schedules overlap or use the same cron expression, they will run serially (one after another). If you require overlapping schedules, you must use a dedicated container for each schedule.
    • Applying Changes: You must manually restart the container for configuration changes to take effect.
    services:
  backup:
    image: offen/docker-volume-backup:v2
    volumes:
      - data:/backup/my-app-backup:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./configuration:/etc/dockervolumebackup/conf.d

volumes:
  data:

  3. Stop containers during backup without restarting

    main

    If you need to stop containers for a backup but do not want them to restart automatically (for example, if they are managed by an external scheduler), use the docker-volume-backup.stop-during-backup-no-restart label.

    This label is mutually exclusive with docker-volume-backup.stop-during-backup. To use a custom label for this behavior, set the BACKUP_STOP_DURING_BACKUP__NO_RESTART_LABEL environment variable in the backup container and match its value in the target container's labels.

    services:
  app:
    labels:
      - docker-volume-backup.stop-during-backup-no-restart=service2

  backup:
    image: offen/docker-volume-backup:v2
    environment:
      BACKUP_STOP_DURING_BACKUP__NO_RESTART_LABEL: service2
    volumes:
      - data:/backup/my-app-backup:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro

volumes:
  data:

  4. Stop services by scaling replicas in Docker Swarm

    main

    To ensure backup integrity in Docker Swarm, you can instruct docker-volume-backup to scale a service down to zero replicas before the backup starts and scale it back up to its original count once the backup is complete.

    Requirements:

    • Docker must be running in Swarm mode.
    • The docker-volume-backup container must be placed on a manager node.
    • This approach only works for services deployed in replicated mode.
    • Use the docker-volume-backup.stop-during-backup=true label within the deploy section of your service definition.
    services:
  app:
    image: myorg/myimage:latest
    deploy:
      labels:
        - docker-volume-backup.stop-during-backup=true
      replicas: 2

  6. Replace deprecated exec-pre and exec-post labels

    main

    Starting from version 2.19.0, the docker-volume-backup.exec-pre and docker-volume-backup.exec-post labels are deprecated in favor of more descriptive terminology. To maintain existing behavior, rename these labels to docker-volume-backup.archive-pre and docker-volume-backup.archive-post respectively.

    Note that the EXEC_LABEL environment variable and the docker-volume-backup.exec-label label are NOT deprecated and should remain unchanged.

        labels:
-     - docker-volume-backup.exec-pre=cp -r /var/my_app /tmp/backup/my-app
+     - docker-volume-backup.archive-pre=cp -r /var/my_app /tmp/backup/my-app
-     - docker-volume-backup.exec-post=rm -rf /tmp/backup/my-app
+     - docker-volume-backup.archive-post=rm -rf /tmp/backup/my-app

  7. Set up recurring backups using Docker Compose

    main

    To implement recurring backups, add a backup service to your docker-compose.yml. You must mount the target volumes as read-only (:ro) to the backup container and configure settings via an environment file.

    Key Configuration Details:

    • Backup Integrity: To ensure data consistency, add the label docker-volume-backup.stop-during-backup=true to the service consuming the volume. This allows the backup tool to stop the container before starting the backup.
    • Docker Socket: Mount /var/run/docker.sock:/var/run/docker.sock:ro to allow the tool to stop and restart containers. If using a proxy, set the DOCKER_HOST environment variable.
    • Local Archives: Mount a local directory or volume to /archive to keep a local copy of the backups. You can change this destination by setting the BACKUP_ARCHIVE environment variable.
    • Configuration: Use an env_file to manage backup settings (e.g., S3 credentials, rotation policies).
    services:
  volume-consumer:
    build:
      context: ./my-app
    volumes:
      - data:/var/my-app
    labels:
      - docker-volume-backup.stop-during-backup=true

  backup:
    image: offen/docker-volume-backup:latest
    restart: always
    env_file: ./backup.env
    volumes:
      - data:/backup/my-app-backup:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /path/to/local_backups:/archive
volumes:
  data:

  8. Isolate custom commands using EXEC_LABEL

    main

    If you have multiple docker-volume-backup containers or multiple backup schedules, use EXEC_LABEL to ensure specific backup instances only trigger commands on the intended containers.

    1. Set the EXEC_LABEL environment variable in your docker-volume-backup service.
    2. Add the docker-volume-backup.exec-label label to the target container, matching the value of EXEC_LABEL.
    services:
  database:
    image: mariadb
    volumes:
      - backup_data:/tmp/backups
    labels:
      - docker-volume-backup.archive-pre=/bin/sh -c 'mysqldump --all-databases > /tmp/volume/dump.sql'
      - docker-volume-backup.exec-label=database

  backup:
    image: offen/docker-volume-backup:v2
    environment:
      EXEC_LABEL: database
    volumes:
      - data:/backup/dump:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro

volumes:
  backup_data:

  9. Configure memory limits for backup services in Swarm

    main

    When running docker-volume-backup in Swarm mode, it is recommended to set a hard memory limit on the backup service. While ~25MB is typically sufficient, you may need to increase this limit if you are backing up very large files (e.g., larger than 500MB) to prevent the process from being Killed.

    services:
  backup:
    image: offen/docker-volume-backup:v2
    deployment:
      resources:
        limits:
          memory: 25M

  11. Deprecated method: Set time zone via bind-mounting host files

    main

    Bind-mounting files from the host (such as /etc/localtime) to set the time zone is deprecated. It is recommended to use the TZ environment variable instead. If you must use this method, you would mount the following files:

    • /etc/timezone
    • /etc/localtime
    • /usr/share/zoneinfo

    Note that if the TZ environment variable is present, it will override these bind-mounted files.

    services:
  backup:
    image: offen/docker-volume-backup:v2
    volumes:
      - data:/backup/my-app-backup:ro
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
      - /usr/share/zoneinfo:/usr/share/zoneinfo:ro

volumes:
  data:

  12. Encrypt backups using GPG

    main

    To encrypt backups with GPG, set one of the following environment variables:

    • GPG_PASSPHRASE: Uses a passphrase for symmetric encryption.
    • GPG_PUBLIC_KEY_RING: Uses a public key ring for asymmetric encryption.

    When encryption is active, the backup archive is saved with a .gpg extension. To decrypt the backup, use the gpg command line tool.