tusd

tusd is the official reference implementation of the tus resumable upload protocol. It allows for HTTP-based file uploads that can be interrupted and resumed without re-uploading previously sent data, making it resilient to network issues or server outages.

Key Features

• Large File Support: Handles arbitrarily large files.
• Client Compatibility: Works with any tus-compatible client.
• Single Binary: Distributed as a standalone binary that does not require a runtime.
• Extensibility: Fully customizable via hooks (Scripts, HTTP, or gRPC) for tasks like:
  • Upload validation
  • User authentication and authorization
  • File post-processing
• Storage Backends: Supports multiple storage options including:
  • Local disk
  • AWS S3
  • Azure Blob Storage
  • Google Cloud Storage
• Go Integration: Can be embedded directly into Go applications.

This example demonstrates how to set up and run a gRPC hook server using Python to interact with tusd. Follow these steps to prepare the environment, build the necessary gRPC code, start the server, and connect tusd to it.

1. Setup virtual environment and install dependencies

Create a virtual environment and install the required Python packages:

python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt

2. Build gRPC code

If the gRPC code is not already generated, use make to build hook_pb2.py:

make -B hook_pb2.py

3. Start the gRPC server

Run the server script. By default, it listens on localhost:8000:

python3 server.py

4. Connect tusd to the gRPC server

In a separate terminal, run tusd and use the -hooks-grpc flag to point it to your running gRPC server:

tusd -hooks-grpc localhost:8000

# Setup virtual environment and install dependencies
python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt

# Build gRPC code, if necessary
make -B hook_pb2.py

# Start gRPC server (listening at localhost:8000)
python3 server.py

# In a separate terminal you can now run tusd and point it to the gRPC server
tusd -hooks-grpc localhost:8000

File hooks allow tusd to execute scripts or executables from a specific directory when an event is triggered.

Setup:

1. Enable file hooks by passing the -hooks-dir flag with the path to your hook directory.
2. Name your executable files exactly after the event they should handle (e.g., pre-create).

Platform Specifics:

• UNIX: Hook files must not have an extension (e.g., use pre-create, not pre-create.sh). Use a shebang (e.g., #!/usr/bin/env python3) to specify the interpreter.
• Windows: Hook files must have an extension (e.g., .bat or .exe).

Execution Environment:

• Environment Variables: TUS_ID, TUS_OFFSET, and TUS_SIZE are provided. Note that TUS_ID is an empty string during the pre-create hook.
• Stdin: A JSON-encoded hook request is available on stdin.
• Stdout: If the process exits with code 0, tusd parses stdout as a JSON-encoded hook response to customize the HTTP response or abort uploads.
• Stderr: Redirected to tusd's stderr for logging.

Error Handling:

• A non-zero exit code is treated as an internal failure. For pre-create and pre-finish, tusd will respond to the client with a 500 Internal Server Error and stop processing.

$ tusd -hooks-dir ./path/to/hooks/