ZipStream-PHP

repository·main·Indexed 23 days ago

https://github.com/maennchen/zipstream-php

A fast and simple streaming ZIP file downloader for PHP that allows streaming archives directly to users or custom streams without writing to disk first. Supports adding files from strings, local paths, stream resources, and PSR-7 streams. Includes features for S3 multipart uploads, Flysystem integration, and simulation modes for calculating Content-Length headers. Requires PHP 8.1+ for version 3.0.0 and PHP 8.2+ for version 3.1.2.

Tokens
11.6K
Snippets
21
Records
37
Agent score
83%

What's inside zipstream-php

  1. Stream ZIP data to a callback function using CallbackStreamWrapper

    main

    The ZipStream\Stream\CallbackStreamWrapper::open() method allows you to intercept the ZIP data stream via a callback function. This is useful for:

    • Streaming to multiple destinations simultaneously (e.g., echoing to the browser while writing to a file).
    • Progress tracking by monitoring the length of the $data chunks.
    • Custom logging or data transformation.

    Note: When using a custom outputStream via a callback, you should set sendHttpHeaders: false in the ZipStream constructor to prevent the library from automatically sending HTTP headers, as you are now manually controlling the output flow.

    Important Recommendation: For data transformations (like encoding), prefer using PHP's built-in stream filters via stream_filter_append() rather than performing transformations inside the callback. Stream filters are more efficient and maintain better data integrity.

    use ZipStream\ZipStream;
    use ZipStream\Stream\CallbackStreamWrapper;
    
    // Example: Stream to multiple destinations
    $backupFile = fopen('backup.zip', 'wb');
    $logFile = fopen('transfer.log', 'ab');
    
    $zip = new ZipStream(
        outputStream: CallbackStreamWrapper::open(function (string $data) use ($backupFile, $logFile) {
            // Send to browser
            echo $data;
            
            // Save to file efficiently
            fwrite($backupFile, $data);
            
            // Log transfer progress
            fwrite($logFile, "Transferred " . strlen($data) . " bytes\n");
        }),
        sendHttpHeaders: false,
    );
    
    $zip->addFile('hello.txt', 'Hello World!');
    $zip->finish();
    
    fclose($backupFile);
    fclose($logFile);
  2. Handle File Path Collisions in Client Code

    main
    Because ZipStream-PHP streams data directly, the library cannot determine if files being added will result in duplicate paths within the archive. It is the responsibility of the implementing client code to ensure that files are not saved with the same path to avoid collisions.
  3. Disable Nginx response buffering for streaming

    main

    When using Nginx as a webserver, it may attempt to buffer the response, which interferes with the streaming behavior of ZipStream-PHP. To prevent this, you should disable buffering by sending the X-Accel-Buffering: no header.

    This can be done using the native PHP header() function or via the Symfony Response object.

  4. Use ZipStream with Symfony StreamedResponse

    main

    When using ZipStream within Symfony controller actions, you must wrap your ZipStream logic inside Symfony's StreamedResponse. This ensures that the output is streamed correctly to the user's browser and prevents the generation of corrupted ZIP files.

    To implement this, pass a callback to the StreamedResponse constructor. Inside this callback, initialize your ZipStream\ZipStream instance and call its streaming methods (such as addFile, addFileFromPath, or addFileFromStream). Finally, call $zip->finish() to complete the archive within the callback.

    use Symfony\Component\HttpFoundation\StreamedResponse;
    use ZipStream\ZipStream;
    
    $response = new StreamedResponse(function() {
        $zip = new ZipStream\ZipStream(
            outputName: 'test.zip',
            defaultEnableZeroHeader: true,
            contentType: 'application/octet-stream',
        );
    
        // ... add files using $zip->addFile() etc.
    
        $zip->finish();
    });
    
    return $response;
  5. Compress large files to S3 compatible storages

    main

    S3 compatible storages typically have a 5 GiB limit for single-part uploads. To generate and upload ZIP archives larger than this, you must use a multi-part upload strategy.

    This is achieved by implementing a PSR-7 stream that buffers ZipStream's output and uploads it to S3 in chunks (parts). This prevents memory exhaustion and bypasses the single-upload size limitation.

  6. Install ZipStream-PHP via Composer

    main

    To add ZipStream-PHP to your project, use Composer to require the package.

    If you intend to use addFileFromPsr7Stream (which requires Psr\Http\Message\StreamInterface) or if you want to use a stream instead of a resource as the outputStream, you must also install psr/http-message and guzzlehttp/psr7.

    If you encounter errors regarding the missing mbstring extension, you can install the Symfony polyfill to resolve the requirement.

  7. Configure Nginx FastCGI buffer parameters

    main
    If you prefer not to use the X-Accel-Buffering header, you can alternatively adjust the fastcgi_cache parameters within your Nginx configuration to accommodate streaming responses. Refer to the official Nginx documentation for ngx_http_fastcgi_module regarding fastcgi_buffers for specific tuning details.
  8. Add files from PSR-7 streams using addFileFromPsr7Stream

    main

    If you are working with PSR-7 compliant streams (such as those provided by Slim, Guzzle, or other PSR-7 middleware), you can add them directly to a ZipStream-PHP archive using the addFileFromPsr7Stream method. This allows you to include content from a PSR-7 response body or any other PSR-7 stream as a file within the ZIP archive without manual buffering.

    $stream = $response->getBody();
    // add a file named 'streamfile.txt' from the content of the stream
    $zip->addFileFromPsr7Stream(
        fileName: 'streamfile.txt',
        stream: $stream,
    );
  9. Configure Varnish to support large Zip stream downloads

    main

    When serving large ZIP files through Varnish, the cache may cause random stream closures. To prevent this, you must configure Varnish to use pipe mode for specific file extensions. This allows the stream to pass through Varnish without being intercepted or buffered, which is essential for the streaming behavior of ZipStream-PHP.

    Add the following configuration to your Varnish VCL file to intercept requests for .tar, .gz, .zip, .7z, and .exe files and pipe them directly.

        sub vcl_recv {
            # Varnish can’t intercept the discussion anymore
            # helps for streaming big zips
            if (req.url ~ "\.(tar|gz|zip|7z|exe)$") {
                return (pipe);
            }
        }
        # Varnish can’t intercept the discussion anymore
        # helps for streaming big zips
        sub vcl_pipe {
            set bereq.http.connection = "close";
            return (pipe);
        }
  10. Add Content-Length header to ZipStream output

    main

    To provide a Content-Length header for a ZIP stream, you must use a two-pass approach called 'Simulation'. This involves configuring the ZipStream with an operationMode of either OperationMode::SIMULATE_STRICT or OperationMode::SIMULATE_LAX.

    1. SIMULATE_STRICT: Prevents the library from reading the entire file to calculate size. This is useful for ensuring that size calculation remains efficient and doesn't accidentally consume large amounts of memory or time by reading unbuffered data.
    2. SIMULATE_LAX: Allows the library to read the entire file if necessary to determine the size.

    Workflow:

    1. Initialize ZipStream with the chosen simulation mode.
    2. Add files to the stream (use addFileFromCallback with exactSize for deferred file opening).
    3. Call $zip->finish() to calculate the resulting ZIP file size.
    4. Send the Content-Length header to the client.
    5. Call $zip->executeSimulation() to perform the actual streaming of the ZIP data.
    use ZipStream\OperationMode;
    use ZipStream\ZipStream;
    
    $zip = new ZipStream(
        operationMode: OperationMode::SIMULATE_STRICT, // or SIMULATE_LAX
        defaultEnableZeroHeader: false,
        sendHttpHeaders: true,
        outputStream: $stream,
    );
    
    // Normally add files
    $zip->addFile('sample.txt', 'Sample String Data');
    
    // Use addFileFromCallback and exactSize if you want to defer opening of
    // the file resource
    $zip->addFileFromCallback(
        'sample.txt',
        exactSize: 18,
        callback: function () {
            return fopen('...');
        }
    );
    
    // Read resulting file size
    $size = $zip->finish();
    
    // Tell it to the browser
    header('Content-Length: '. $size);
    
    // Execute the Simulation and stream the actual zip to the client
    $zip->executeSimulation();