adm-zip

The Deflate algorithm stores compressed data in blocks. Each block starts with a header that defines its type. When parsing or implementing Deflate, identify the block type using bits 1-2 of the header:

• 00 (0): Stored Block. Data is uncompressed and byte-aligned. The block contains a length word followed by its one's complement.
• 01 (1): Fixed Huffman Codes. Uses a predefined set of Huffman codes for literals and distances.
• 10 (2): Dynamic Huffman Codes. Uses custom Huffman codes defined within the block header.
• 11 (3): Reserved. Encountering this should trigger an "Error in compressed data".

Additionally, Bit 0 of the header is the Last Block bit, which is set to 1 if the current block is the final one in the stream.

Applications may use Manifest Files to store additional information required by the application process that cannot be stored using standard ZIP storage records.

Key characteristics:

• Location: Typically placed within the same ZIP file as the data it describes. By convention, it is often the first file in the archive and may include a defined directory path.
• Format: Can be any file type required by the application (e.g., META-INF/MANIFEST.MF in .JAR files).
• Compression: Manifest files may be compressed or encrypted.

Note: Manifest file specifications are outside the scope of the standard ZIP specification.

WavPack compression (Method 97) uses the open-source WavPack utility.

Implementation Details:

• Header Indication: The compression method field must be set to 97 in both the local header and the central directory header.
• Data Placement: WavPack data begins immediately after the end of the local header data.
• Digital Sample Data Best Practice: When storing digital sample data, compress all bytes of the sample data, including unused bits up to the byte boundary. For example, if a 2-byte sample uses only 12 bits, provide the full 16-bit sample size to the WavPack routines to prevent unused bits from being zeroed out during extraction.

LZMA compression in a ZIP file consists of an LZMA Properties Header followed by the LZMA Compressed Data.

LZMA Properties Header Structure:

• LZMA Version Information (2 bytes): First byte is the major version, second byte is the minor version of the LZMA SDK used.
• LZMA Properties Size (2 bytes): Defines the size of the following property data.
• LZMA Property Data (variable): Required values for decompression (e.g., dictionary size).

Key Implementation Details:

• EOS Marker: LZMA data may include an end-of-stream (EOS) marker. If present, general purpose bit 1 must be set in the ZIP header. If bit 1 is not set, the EOS marker is assumed to be absent.

A .ZIP archive is composed of a series of repeating file segments followed by a central directory. The standard sequence for a file entry is:

1. Local File Header: Contains metadata about the file.
2. Encryption Header (Optional): Present if the file is encrypted.
3. File Data: The actual compressed or stored content.
4. Data Descriptor (Optional): Used for streaming; follows the file data.

After all file entries, the archive contains:

• Archive Decryption Header (Optional): Used for Central Directory Encryption.
• Archive Extra Data Record (Optional).
• Central Directory Headers: One for each file in the archive.
• Zip64 End of Central Directory Record (if applicable).
• Zip64 End of Central Directory Locator (if applicable).
• End of Central Directory Record.

The ZIP specification supports several strong encryption algorithms. When working with encrypted archives, you may encounter different AlgId values representing these algorithms:

• AES: 128 bit (0x660E), 192 bit (0x660F), and 256 bit (0x6610)
• 3DES: 168 bit (0x6603) and 112 bit (0x6609)
• DES: (0x6601)
• RC2: Version-dependent (0x6602 for < 5.2, 0x6702 for >= 5.2)
• Blowfish: (0x6720)
• Twofish: (0x6721)
• RC4: (0x6801)

ZIP files can be segmented into multiple parts using two different methods: Spanning and Splitting.

Spanned ZIP Files

Used typically for removable media (like floppy disks). All segments share the same filename. The sequence is determined by the DOS volume label in the form PKBACK#xxx, where xxx is the decimal segment number (e.g., 001 to nnn).

Split ZIP Files

Used for local file systems to avoid name collisions. Segments are named using a specific sequence:

• Segment 1: filename.z01
• Segment n-1: filename.z(n-1)
• Final Segment: filename.zip (The .ZIP extension is used on the last segment to allow quick reading of the central directory).

Split PKSFX (Self-extracting) Files

If a split archive is a PKSFX self-extracting file, the first segment must be named filename.exe and must be large enough to contain the entire executable program.

When working with ZIP files that require Unicode support, the UnicodeName is the UTF-8 version of the File Name field.

To indicate that both the File Name and Comment fields are UTF-8, use the General Purpose Bit Flag, bit 11 (Language encoding flag (EFS)).

Important constraints:

• No UTF-8 Byte Order Mark (BOM) should be used.
• If bit 11 is used, the Unicode Path and Unicode Comment extra fields are not needed and should not be created.
• For backward compatibility, bit 11 should only be used if the native character set of the paths and comments being zipped is already UTF-8.
• The same storage method (either bit 11 or extra fields) must be used consistently in both the Local and Central Directory Header for a file.