blockdiff

Use the create command to generate a .bdiff file representing the differences between a target.img and a base.img. This is useful for VM disk images or other large files where you want to store only the changes relative to a known base image.

blockdiff create output.bdiff target.img --base base.img

You can use blockdiff to consolidate sparse files (files that have a large logical size but contain relatively little actual data) into a compact .bdiff file. This is useful for reducing storage costs when uploading to cloud storage.

To compactify a file, run create without the --base flag. To restore the sparse file from the compact version, use apply.

# Consolidate sparse file into a new compact blockdiff
blockdiff create compact.bdiff target.img

# Create a new sparse target.img from the blockdiff
blockdiff apply compact.bdiff target.img

Use the apply command to reconstruct a target.img by combining an existing .bdiff snapshot with its corresponding base.img.

blockdiff apply input.bdiff target.img --base base.img

A .bdiff file follows this structure:

1. Header: Serialized using bincode containing:
   • magic: 8 bytes (BDIFFv1\0)
   • target_size: 8 bytes (u64, little-endian)
   • base_size: 8 bytes (u64, little-endian)
   • ranges: A list of DiffRange objects, where each range contains:
     • logical_offset: 8 bytes (u64, little-endian)
     • length: 8 bytes (u64, little-endian)
2. Padding: Zero-padding to align the start of the data to the next 4 KiB (4096 bytes) boundary.
3. Range Data: Contiguous blocks of data corresponding to the ranges defined in the header.

The blockdiff tool relies on fiemap to inspect file extents. It will error if it encounters extents with the following flags, as they indicate an unstable or unsupported state:

• DELALLOC: File has pending delayed allocations. (Action: sync the file).
• UNWRITTEN: File has unwritten extents. (Action: sync the file).
• NOT_ALIGNED: File has unaligned extents. (Action: sync the file).
• UNKNOWN: Data location is unknown.
• ENCODED: File contains encoded data.
• DATA_ENCRYPTED: File contains encrypted data.

Use the create command to generate a .bdiff file representing the differences between a target file and an optional base file. The tool identifies blocks that have changed (either by being non-shared extents or having different physical locations) and stores only those ranges. If no base file is provided, the tool treats the entire target file as the diff.

blockdiff create <bdiff_output> <target_file> [--base <base_file>]

Use the view command to inspect the metadata of a .bdiff file or the physical extents of a standard file.

• For .bdiff files: Displays the magic string, target/base sizes, number of ranges, and the specific logical offsets and lengths of the diff ranges.
• For regular files: Displays the total file size and the logical/physical extents (using fiemap).

Filtering: You can provide an optional hex offset (e.g., 0x1000) to filter the output. The tool will show the range/extent containing that offset, plus three ranges/extents before and after it.

blockdiff view <bdiff_input> [offset]

# Example: View ranges around offset 0x5000
blockdiff view my_diff.bdiff 0x5000

Use the apply command to reconstruct a target file using a .bdiff file and an optional base file.

• With a base file: The target file is initialized as a reflink copy of the base file, and then the diff ranges are applied to overwrite the necessary blocks. This is highly efficient.
• Without a base file: The target file is created as an empty sparse file of the required size, and the diff ranges are written into it.

blockdiff apply <bdiff_input> <target_file> [--base <base_file>]