yauzl

repository·master·Indexed 21 days ago

https://github.com/thejoshwolfe/yauzl

A Node.js library for unzipping files designed to follow the ZIP specification strictly, avoid blocking the event loop, and maintain low memory usage via streaming. It supports opening ZIP files from paths, file descriptors, Buffers, or custom random access readers, and provides both callback-based and Promise-based APIs (including async iterators via eachEntry()).

Tokens
7.8K
Snippets
23
Records
39
Agent score
24%

What's inside yauzl

  1. Implement a custom RandomAccessReader

    master

    To use fromRandomAccessReader(), you must subclass RandomAccessReader and implement the _readStreamForRange(start, end) method.

    Implementation Requirements:

    • start and end are Number byte offsets.
    • end is exclusive.
    • The method must return a ReadableStream that can be pipe()ed.
    • The stream should provide data in chunks. If the stream provides too many or too few bytes for the requested range, an error will be emitted.
    class MyReader extends RandomAccessReader {
      _readStreamForRange(start, end) {
        // Return a readable stream for the range [start, end)
        // ...
      }
    }
  2. Understand yauzl limitations and constraints

    master

    Before using yauzl, be aware of the following technical limitations:

    • No Streaming Unzip API: Due to the ZIP specification (where the Central Directory is at the end of the file), yauzl does not support streaming the entire archive from start to finish. It prioritizes correctness over the convenience of a streaming interface.
    • Node.js Version: Requires Node.js version 12 or higher.
    • ZIP64 Support: Supports ZIP64 files up to 8PiB. It does not support the full 16EiB range due to JavaScript's IEEE 754 double precision float limitations.
    • Compression Methods: Only supports method 0 (stored) and method 8 (deflated). Other methods will cause openReadStream() to return an error.
    • Encryption:
      • Traditional Encryption: Can be detected via generalPurposeBitFlag, but yauzl does not decrypt it. Use the decodeFileData option in openReadStream() to manage this.
      • Strong Encryption: Emits an error.
      • Encrypted Central Directory: Emits an error.
    • Multi-Disk Archives: Not supported. If the "number of this disk" field in the End of Central Directory Record is not 0, the opening methods will return an err.
    • Ignored Fields/Features:
      • Local File Headers (ignored to ensure spec compliance).
      • CRC-32 (provided in Entry but not used for validation).
      • versionNeededToExtract.
      • Data Descriptors.
      • Archive Extra Data Records.
      • Language Encoding Flags.
      • ZIP64 Extensible Data Sectors.
  3. Understand ZipFile events

    master

    A ZipFile instance emits several key events during its lifecycle:

    • entry: Emitted when a new entry is found. The callback receives an Entry object. If decodeStrings is true, names are already validated.
    • end: Emitted after the last entry event has been emitted.
    • close: Emitted after the file descriptor (or RandomAccessReader) is actually closed. Note that for fromBuffer(), this event is never emitted.
    • error: Emitted if an error occurs while reading the zip file. Once an error is emitted, no further entry, end, or error events will be emitted, though close may still occur.
  4. How readEntry() works with lazyEntries

    master

    When a ZipFile is created with the lazyEntries: true option, the entry and end events are not emitted automatically. Instead, they are only emitted in response to manual calls to readEntry().

    Constraints:

    • readEntry() must not be called if you are using eachEntry().
    • Calling readEntry() multiple times before the previous response event has been emitted results in undefined behavior.
    • Calling readEntry() after the end event or after close() results in undefined behavior.
  5. Handle stream destruction in openReadStream()

    master

    If you intend to call readStream.destroy() on streams obtained from openReadStream(), the returned stream must implement the ._destroy(err, callback) method as per the Node.js stream API.

    Requirements for ._destroy():

    • It must abort any streaming in progress.
    • It must clean up any associated resources.
    • It will only be called after the stream has been unpipe()d from its destination.

    If you never call readStream.destroy(), the streams returned by this method are not required to implement ._destroy().

  6. How to handle errors and avoid crashing

    master

    By default, yauzl will throw an exception (crash) when encountering a malformed zipfile. To handle errors gracefully, follow these patterns based on the API you are using:

    • Callback-based APIs: Always check the err parameter in the callback.
    • Promise-based APIs: Use try...catch or .catch() to handle promise rejections.
    • Event-based access (Event: "entry" or Event: "end"): Attach a listener for Event: "error" on the ZipFile instance.
    • eachEntry() iteration: If using await or for await...of, wrap the loop in a try...catch block to catch rejections.
    • Entry Read Streams: Streams returned by openReadStream() can emit errors. Attach an error event listener to the stream or handle it via the stream's error handling mechanism.
  7. Unzip files using callbacks

    master

    If you prefer a callback-based approach, use yauzl.open(). When using callbacks, you must set {lazyEntries: true} in the options to manually control the entry reading process via zipfile.readEntry().

    In the entry event listener, check if entry.fileName ends with / to identify directories. For file entries, use zipfile.openReadStream(entry, callback) to obtain a stream. You must call zipfile.readEntry() again after a stream ends (or after processing a directory) to continue reading the next entry.

    var yauzl = require("yauzl");
    
    yauzl.open("path/to/file.zip", {lazyEntries: true}, function(err, zipfile) {
      if (err) throw err;
      zipfile.readEntry();
      zipfile.on("entry", function(entry) {
        if (entry.fileName.endsWith("/")) {
          // Ignore directory entry.
          zipfile.readEntry();
        } else {
          // file entry
          zipfile.openReadStream(entry, function(err, readStream) {
            if (err) throw err;
            readStream.on("end", function() {
              zipfile.readEntry();
            });
            readStream.pipe(somewhere);
          });
        }
      });
    });
  8. Unzip files using async/await

    master

    You can use the Promise-based API to iterate through a ZIP file's entries using await and for await...of. This is the recommended way to handle ZIP files without blocking the JavaScript thread.

    When iterating, directory entries are identified by a trailing slash (/) in their fileName. To extract a file, use openReadStreamPromise(entry) to get a readable stream for that specific entry.

    const yauzl = require("yauzl");
    
    (async () => {
      try {
        const zipfile = await yauzl.openPromise("path/to/file.zip");
        for await (let entry of zipfile.eachEntry()) {
          if (entry.fileName.endsWith("/")) {
            // Directory file names end with '/'.
            continue;
          } else {
            // file entry
            const readStream = await zipfile.openReadStreamPromise(entry);
            await stream.promises.pipeline(readStream, somewhere);
          }
        }
      } catch (err) {
        // Indicates a malformed zipfile or I/O error.
        throw err;
      }
    })();
  9. Get file modification dates with getLastModDate()

    master

    The getLastModDate([options]) method returns a JavaScript Date object representing the file's modification time. It attempts to use high-precision extensions before falling back to DOS formats.

    Supported Encodings (in order of preference):

    1. Info-ZIP "universal timestamp" (0x5455): UTC, 1-second precision.
    2. NTFS extended field (0x000a): UTC, 1-millisecond precision.
    3. DOS lastModFileDate and lastModFileTime: Fallback, 2-second precision.

    Options:

    {
      timezone: "local", // or "UTC"
      forceDosFormat: false,
    }
    • timezone: Only affects the DOS fallback. Use "local" (default) or "UTC".
    • forceDosFormat: If true, ignores universal timestamp and NTFS fields (reverts to pre-3.2.0 behavior).
    // Example usage
    const date = entry.getLastModDate({ timezone: 'UTC' });
  10. Configure file data decoding

    master
    In version 3.3.0, yauzl introduced support for controlling how file data is decoded. You can use the decodeFileData option when calling entry.openReadStream() to manage this behavior. This is associated with the entry.canDecodeFileData() method.
  11. Open a read stream for an entry with openReadStream()

    master

    Use openReadStream(entry, [options], callback) to obtain a Readable Stream for a specific Entry object.

    Options:

    • decodeFileData (boolean, default: true): If true, yauzl attempts to decode file data using a zlib inflate transform. Set to false to get raw bytes.
      • Note: If you specify start or end byte offsets, you should generally set decodeFileData: false unless the entry is uncompressed and unencrypted.
    • start (integer): The inclusive byte offset into the entry's file data.
    • end (integer): The exclusive byte offset into the entry's file data.
    • decompress (deprecated): Use decodeFileData instead.
    • decrypt (deprecated): Use decodeFileData instead.

    Error Handling:

    • The readStream may emit errors if decompression fails (zlib error) or if the actual byte count does not match the uncompressedSize (if validateEntrySizes is enabled).
    • To stop reading, call readStream.destroy(). You must unpipe() the stream before destroying it.
    zipfile.openReadStream(entry, { decodeFileData: true }, (err, readStream) => {
      if (err) throw err;
      readStream.on('data', (chunk) => { /* ... */ });
      readStream.on('end', () => { /* ... */ });
    });
  12. Validate file names to prevent directory traversal attacks

    master

    To protect against malicious file paths (e.g., ../../etc/passwd), use yauzl.validateFileName(fileName).

    It returns null if the name is valid, or a String error message if the name is invalid (e.g., starts with /, contains .., or contains \).

    Usage:

    var errorMessage = yauzl.validateFileName(fileName);
    if (errorMessage != null) throw new Error(errorMessage);

    Note: If decodeStrings is true during open(), yauzl automatically runs this validation for every entry.