AzCopy v10 Documentation

Choosing between copy and sync depends on whether you need a simple transfer or an incremental update.

copy command

• Behavior: A simple transferring operation. It scans/enumerates the source and attempts to transfer every single file/blob present on the source to the destination.
• Use Case: Best for moving files when performance is the priority and you don't need to check for existing files.
• Overwriting: By default, AzCopy will overwrite files at the destination if they already exist. Use --overwrite=false to prevent this.

sync command

• Behavior: Scans/enumerates both the source and the destination to find incremental changes. It ensures that whatever is present in the source is replicated to the destination.
• Use Case: Best for incremental transfers (only modified or missing files are transferred).
• Performance: Relatively slower than copy because it must enumerate both sides.
• Overwriting: By default, sync uses the last-modified-time. If the source file is newer than the destination file, it overwrites the destination. Use --mirror-mode=true to change this behavior.
• Deletion: By default, sync does not delete files in the destination that no longer exist in the source. You must use an optional flag to enable deletion.

You can run AzCopy using a Docker container pulled from the Azure Container Registry (azcopycontainers.azurecr.io). This requires authenticating with Azure and the specific registry before pulling and running the image.

Prerequisites

1. Azure CLI installed: You must have the az CLI available.
2. Docker installed: You must have a Docker runtime available on your machine.
3. Permissions: You need access to the azcopycontainers registry.

Workflow

1. Authenticate with Azure

Log in to your Azure account using the Azure CLI:

az login

2. Authenticate with the Azure Container Registry (ACR)

Log in to the specific registry hosting the AzCopy images:

az acr login --name azcopycontainers

3. Locate the Image

List the available repositories in the registry to find the correct <imagename> and <tag>:

az acr repository list --name azcopycontainers --output table

4. Pull the Image

Download the desired image to your local machine:

docker pull azcopycontainers.azurecr.io/<imagename>:<tag>

5. Run AzCopy via Docker

To execute an AzCopy command, run the container with a volume mount to allow AzCopy to access your local files. Use the --rm flag to automatically remove the container after the command completes.

Command Pattern:

docker run --rm -it -v /local/path/to/mount:/azcopy azcopycontainers.azurecr.io/<imagename>:<tag> azcopy copy <source> <destination>

Note on Volume Mounting: The -v /local/path/to/mount:/azcopy flag maps a directory on your host machine to the /azcopy directory inside the container. Ensure your <source> or <destination> paths in the azcopy command are relative to or absolute within the container's filesystem (e.g., /azcopy/mydata).

To have AzCopy automatically manage and display progress for a background job, you must implement the WorkController interface and pass it to InitiateProgressReporting().

The WorkController interface requires:

• Cancel(mgr LifecycleMgr): Called when a cancellation signal (like Ctrl+C or cancel via stdin) is received. Use this to trigger your job's cleanup.
• ReportProgressOrExit(mgr LifecycleMgr) (totalKnownCount uint32): Called periodically by the manager. It should return the current count of processed items. If the work is complete, the implementation can signal the manager to exit.

Lifecycle Note: If you are running cleanup jobs after a primary job, you must call AllowReinitiateProgressReporting() before starting the next progress reporting cycle.

type MyWorkController struct { /* ... */ }

func (w *MyWorkController) Cancel(mgr cmd.LifecycleMgr) {
    // logic to stop the actual transfer/work
}

func (w *MyWorkController) ReportProgressOrExit(mgr cmd.LifecycleMgr) uint32 {
    // logic to get current progress
    // return count of items processed
    return currentCount
}

// Usage:
lcm := cmd.GetLifecycleMgr()
lcm.InitiateProgressReporting(&MyWorkController{})

To access Azure Storage resources using Microsoft Entra ID, you must first log in. Ensure your user account has the Storage Blob Data Contributor role assigned.

Authentication Methods:

• Interactive: azcopy login (uses default tenant).
• Specific Tenant: azcopy login --tenant-id "[TenantID]".
• Managed Identity (VM): azcopy login --identity.
• Service Principal (Secret): Set AZCOPY_SPA_CLIENT_SECRET and use azcopy login --service-principal --application-id <app-id>.
• Service Principal (Certificate): Use azcopy login --service-principal --certificate-path /path/to/cert --application-id <app-id>. (Requires AZCOPY_SPA_CERT_PASSWORD env var).
• Device Login: Set AZCOPY_AUTO_LOGIN_TYPE=DEVICE and run azcopy login to get a code for browser authentication.
• Managed Identity (MSI): Set AZCOPY_AUTO_LOGIN_TYPE=MSI. Use AZCOPY_MSI_CLIENT_ID or AZCOPY_MSI_RESOURCE_STRING as needed.

Check status with azcopy login status and terminate access with azcopy logout.

# Interactive login
azcopy login

# Service Principal login
export AZCOPY_SPA_CLIENT_SECRET="your-secret"
azcopy login --service-principal --application-id "your-app-id"