BorgBackup

Borg supports two primary methods for storing encryption keys and secrets:

1. keyfile mode: The key is stored in a file within the keys/ subdirectory of the Borg configuration directory on the client.
2. repokey mode: The key is stored within the repository itself under the keys/ namespace, named using the SHA256 of the Borg key content.

Commonalities:

• Both modes use the same internal data format.
• Keys are generated from random data and then encrypted using a key derived from your passphrase.
• Borg identifies the correct key by attempting to decrypt it with the provided passphrase.
• The passphrase can be provided via the BORG_PASSPHRASE environment variable or through interactive prompts.

Repository object IDs (used as keys in the key-value store) are 256-bit (32-byte) strings. They are computed by applying an id_hash function to the plaintext data (before encryption, compression, or obfuscation).

The id_hash function is selected during repository creation using the --id-hash flag. For encrypted repositories, this is a keyed MAC over the plaintext using the id_key:

• sha256: Uses HMAC-SHA256.
• blake3: Uses a keyed BLAKE3.
• none: Uses a plain SHA256 (only for unencrypted repositories).

Because these IDs are used for deduplication, the selected hash must be cryptographically strong.

borg repo-create --id-hash sha256 ...

To locate a chunk within a pack, Borg uses a ChunkIndex. The index maps a chunk_id to a specific location within a pack file.

Mapping: chunk_id $\rightarrow$ (flags, size, pack_id, obj_offset, obj_size)

• obj_offset: The byte offset of the blob from the start of the pack file.
• obj_size: The total blob length (header + encrypted_meta + encrypted_data).
• size: The plaintext chunk size.
• F_PENDING flag: Indicates a chunk is currently being buffered in the pack writer but its pack location is not yet resolved.

Retrieval: A reader fetches a chunk using a single range request: read packs/<hex(pack_id)> at [obj_offset, obj_offset + obj_size).

Each repository object is stored separately under its ID in the data/ directory. An object consists of:

1. 32-bit meta size
2. 32-bit data size
3. meta: Metadata (MessagePack-encoded and separately encrypted/authenticated).
4. data: The actual payload (encrypted and compressed).

Metadata is stored separately to allow efficient querying (reading/decrypting only the small metadata part) without needing to process the much larger data payload.

The files cache uses an age value for lifecycle management.

• When a file is seen during a backup run, its age is reset to 0.
• If a file is not seen, its age is incremented by one.
• Once a file's age reaches the value defined by BORG_FILES_CACHE_TTL, its entry is removed from the cache.

To minimize memory overhead, Borg stores the files cache in memory as a compressed Python dictionary where chunk lists are stored as 32-bit indexes into the chunks index rather than full 256-bit IDs.

Borg uses unambiguous Message IDs to identify log messages and operations. This allows developers to programmatically handle specific errors or events without parsing variable text strings. Each Message ID is associated with a specific Return Code (rc), which serves as the process exit code.

When developing frontends or automation scripts, you should rely on these IDs and exit codes for robust error handling.

Key categories include:

• Errors (rc 2-92): Critical failures (e.g., Repository.DoesNotExist rc 13, LockTimeout rc 73).
• Warnings (rc 1, 100-107): Non-fatal issues (e.g., IncludePatternNeverMatchedWarning rc 101).
• Operations: Specific task names like archive.delete or repository.check used in internal RPC/logic.

All Borg commands support a set of common options. While the specific list of options is defined in the common options documentation, a key behavior to note is how the --help flag interacts with subcommands:

• For subcommands (e.g., borg help compact), the --help flag works as expected.
• For sub-sub-commands (e.g., borg help key export), the --help flag may not work when used with the help command.
• Workaround: Use the help command as a flag directly on the sub-sub-command (e.g., borg key export --help).

As of version 0.28.0, BorgBackup uses the following exit codes to indicate the status of a command:

• 0: Success (OK)
• 1: Warning
• 2: Error

Prior to version 0.28.0, the codes were 0 for success and 1 for error.

When working with remote Borg servers, you can control which Borg version is used by setting the BORG_VERSION environment variable before executing the borg serve command via SSH.

Additionally, Borg supports placeholders in the --remote-path option, such as {borgversion}, which can be used to facilitate version-specific remote path configurations.