Flow.js Documentation

This sample illustrates how a PHP server can receive file chunks sent by the Flow.js client.

Key behaviors:

• Chunk Storage: It receives chunks via standard multipart/form-data and stores them in a temp/ directory using the naming convention: [flowFilename].part[flowChunkNumber].
• GET Requests: It handles GET requests to verify if a specific chunk exists (used for test() calls), returning 200 OK if the chunk is present and 404 Not Found otherwise.
• File Assembly: Once all parts are present (calculated by comparing flowChunkSize and flowTotalSize against the number of parts found), it appends the parts into a single final destination file.

⚠️ Security Warning: This script is for educational purposes only. It does not sanitize file names. You must implement cleaning for dots and dashes to prevent directory traversal attacks (files escaping the temporary upload directory).

<?php
// ... (See full implementation in source for logic details)

// Example of how the script handles GET requests for chunk testing:
if ($_SERVER['REQUEST_METHOD'] === 'GET') {
    $temp_dir = 'temp/'.$_GET['flowIdentifier'];
    $chunk_file = $temp_dir.'/'.$_GET['flowFilename'].'.part'.$_GET['flowChunkNumber'];
    if (file_exists($chunk_file)) {
         header("HTTP/1.0 200 Ok");
       } else {
         header("HTTP/1.0 404 Not Found");
       }
}

Create a new Flow object by specifying the target (the upload endpoint) and an optional query object for additional parameters. You can check flow.support to provide a fallback for unsupported browsers.

To enable file selection via a button or drag-and-drop, use assignBrowse() and assignDrop().

Interaction is handled via event listeners like fileAdded, fileSuccess, and fileError.

var flow = new Flow({
  target:'/api/photo/redeem-upload-token', 
  query:{upload_token:'my_token'}
});

// Check for support
if(!flow.support) location.href = '/some-old-crappy-uploader';

// Assign UI elements
flow.assignBrowse(document.getElementById('browseButton'));
flow.assignDrop(document.getElementById('dropTarget'));

// Listen to events
flow.on('fileAdded', function(file, event){
    console.log(file, event);
});
flow.on('fileSuccess', function(file,message){
    console.log(file,message);
});
flow.on('fileError', function(file, message){
    console.log(file, message);
});

This guide provides a Haskell implementation pattern for a resumable file upload backend, compatible with Flow.js client-side logic. The implementation involves three primary stages: initializing the upload, receiving file chunks, and verifying chunk integrity.

1. Initialize Upload (uploadStart)

When a client starts an upload, the server must:

• Validate the filename and file size.
• Create an upload token/identifier.
• Pre-allocate the file on disk using setFdSize to the expected total size.
• Return the upload token to the client.

2. Handle File Chunks (uploadChunk)

For each chunk sent by the client, the server should:

• Validate the chunk metadata (identifier, filename, total size, chunk size, chunk number, and current chunk size).
• Open the target file and seek to the correct offset (fdSeek) using the calculated offset (off).
• Write the incoming byte stream to the file at that offset.
• Verify that the number of bytes written matches the expected chunk length.

3. Verify Chunk Integrity (testChunk)

To support resumable logic, the server can implement a GET endpoint that:

• Seeks to the specified offset.
• Reads the expected number of bytes from the file.
• Returns a 200 OK if the data is present and non-zero, or 204 No Content if the chunk is empty or missing, allowing the client to decide whether to re-upload.

To build a Go backend for Flow.js, you need to handle three main stages of the chunked upload process:

1. Check for existing chunks (GET request): Before uploading, the client sends a GET request to verify if a specific chunk already exists on the server. If the server returns a 204 No Content (or similar indicating the chunk is missing), the client proceeds with the upload.
2. Receive and save chunks (POST request): The client sends chunks via POST requests. The server must parse the multipart form, extract the flowFilename and flowChunkNumber from the request parameters, and save the file data to a directory specific to that filename.
3. Assemble chunks: Once the number of received chunks matches the flowTotalChunk count, the server should trigger a process to stitch the chunks together in the correct order and delete the temporary chunk files.

Recommended Libraries:

• github.com/patdek/gongflow
• github.com/stuartnelson3/golang-flowjs-upload (includes a streaming handler to reduce memory footprint)

// Example logic for handling chunk existence (GET)
func continueUpload(w http.ResponseWriter, r *http.Request) {
	chunkDirPath := "./incomplete/" + r.FormValue("flowFilename") + "/" + r.FormValue("flowChunkNumber")
	if _, err := os.Stat(chunkDirPath); err != nil {
		w.WriteHeader(204)
		return
	}
}

// Example logic for receiving chunks (POST)
func chunkedReader(w http.ResponseWriter, r *http.Request) error {
	r.ParseMultipartForm(25)

	chunkDirPath := "./incomplete/" + r.FormValue("flowFilename")
	err := os.MkdirAll(chunkDirPath, 02750)
	if err != nil {
		return err
	}

	for _, fileHeader := range r.MultipartForm.File["file"] {
		src, err := fileHeader.Open()
		if err != nil {
			return err
		}
		defer src.Close()

		dst, err := os.Create(chunkDirPath + "/" + r.FormValue("flowChunkNumber"))
		if err != nil {
			return err
		}
		defer dst.Close()
		io.Copy(dst, src)
	
		// Check if all chunks are present to trigger assembly...
	}
	return nil
}

After calling your handle_resumable_file procedure, you must interpret the returned status to decide how to respond to the client. The procedure returns a status string and associated context.

Status Mapping

• partly_done: The upload is in progress. Return 200 OK to the client.
• done: All chunks have been successfully received and collated. You can then proceed to process the final file.
• invalid_resumable_request: The request was malformed or failed integrity checks. Return a 500 Internal Server Error with the error context.
• non_resumable_request: The request was a standard non-resumable HTTP upload. Handle it using your existing standard file upload logic.

    # Example status handling logic
    lassign [handle_resumable_file] resumable_status resumable_context resumable_original_filename resumable_identifier
    if { $resumable_status ne "non_resumable_request" } {
        switch -exact $resumable_status {
            partly_done {
                doc_return 200 text/plain ok
                ad_script_abort
            }
            done {
                # Process the final file
            }
            invalid_resumable_request - {
                doc_return 500 text/plain $resumable_context
                ad_script_abort
            }
            default {
                doc_return 500 text/plain $resumable_context
                ad_script_abort
            }
        }
    }

To handle resumable uploads in AOLserver or OpenACS environments, you can implement a handler using the ad_proc pattern. The handler must process specific parameters sent by the client to manage chunked uploads, verify file integrity, and collate chunks once the upload is complete.

Required Request Parameters

The following parameters must be handled in your ad_page_contract:

• resumableChunkNumber: integer representing the current chunk index.
• resumableChunkSize: integer representing the size of each chunk.
• resumableTotalSize: integer representing the total size of the file.
• resumableIdentifier: a unique string identifying the upload session.
• resumableFilename: the original name of the file.
• file: the actual file data (the parameter name used for the upload).

Implementation Workflow

1. Sanity Check: Validate that resumableChunkNumber, resumableChunkSize, resumableTotalSize, and resumableIdentifier are present and non-zero.
2. Identifier Sanitization: Clean the resumableIdentifier (e.g., using regsub) to ensure it only contains alphanumeric characters, underscores, or hyphens to prevent path traversal or filesystem issues.
3. Handle GET Requests: If the request method is GET, check if the specific chunk file already exists on the server and matches the expected resumableChunkSize. Return 200 OK if it exists, or 204 Not Found if it does not. This allows the client to resume from the last successful chunk.
4. Chunk Storage: Save each incoming chunk to a temporary location using a naming convention like resumable-${resumableIdentifier}.${resumableChunkNumber}.
5. Integrity Verification: Verify that the size of the received chunk matches the expected size for that specific chunk index (especially important for the last chunk, which may be smaller than the standard chunk size).
6. Collation: Once all chunks are present, concatenate them into a single final file using a command like cat and delete the individual chunk files.

    ad_proc handle_resumable_file {
        {-file_parameter_name "file"}
        {-folder "/tmp"}
        -check_video:boolean
        {-max_file_size ""}
    } {}