copyparty

When choosing a file server, consider these key differentiators for copyparty:

Core Advantages

• Resumable & Accelerated Uploads: Uploads are verified server-side, allowing for much faster transfers (especially across long distances) and support for terabyte-sized files even behind Cloudflare.
• High Performance: Significantly faster at handling small files (e.g., ~670 files/sec) compared to alternatives like Nextcloud (4/sec) or Seafile (4.7/sec).
• Portability: Extremely lightweight (standalone Python file ~0.6 MB or Windows EXE ~6 MiB) with no dependencies.
• Filename Support: Handles 'crazy' filenames that some other servers (like hfs3 or dufs) may not support.

Comparison Summary

Deduplication reduces disk usage by using links instead of full copies when a duplicate file is uploaded.

Activation

• Global: Use --dedup.
• Per-volume: Use the dedup volflag.

Deduplication Methods

By default, copyparty uses symlinks. You can change this behavior:

• Hardlinks: Use --hardlink-only or hardlinkonly. Hardlinks behave like regular files and are more compatible with other software, but editing a hardlinked file edits all copies. They also share a single timestamp across all links.
• Reflinks (CoW): Use --reflink or reflink. This is the safest method as it uses Copy-on-Write, allowing files to be edited independently. Requirements: Linux 5.3+ and Python 3.14+; requires compatible filesystems like Btrfs.

Critical Safety Warnings

• Indexing: When enabling deduplication, it is strongly recommended to also enable indexing using -e2dsa or the e2dsa volflag.
• Editing: If you edit the contents of a deduplicated file, you will edit all copies of that file. This is especially true for hardlinks.
• S3/Glacier: If using expensive storage where reading is costly, you may want to disable duplicate detection entirely using --no-clone or the noclone volflag.

[global]
  e2dsa  # scan and index filesystem on startup
  dedup  # symlink-based deduplication for all volumes

[/media]
  /mnt/nas/media
  flags:
    hardlinkonly  # this vol does hardlinks instead of symlinks

Event hooks allow you to trigger external programs or ZeroMQ messages when specific actions occur, such as uploads, renames/moves, or deletes.

ZeroMQ Integration: Instead of running shell programs, hooks can send ZeroMQ messages. This is useful for notifying other services.

• zmq:pub:<address>: Sends a PUB message to all connected SUB clients.
• zmq:push:<address>: Sends a PUSH message to exactly one connected PULL client (use t[seconds] like t3 to avoid blocking if no clients are connected).
• zmq:req:<address>: Sends a REQ message to a connected REP client (use t3,j to send extended upload info as JSON).

Legacy Upload Events: For a more powerful, non-blocking, and multithreaded approach that has access to FFmpeg/mtp tags, you can use the older mte (metadata event) system. This is useful if you want to trigger a command only on new unique files and want to avoid holding up other uploads.

[global]
  xau: zmq:pub:tcp://*:5556`  # send a PUB to any/all connected SUB clients
  xau: t3,zmq:push:tcp://*:5557`  # send PUSH to exactly one connected PULL cli
  xau: t3,j,zmq:req:tcp://localhost:5555`  # send REQ to the connected REP cli

Dirkeys allow sharing specific folders without granting full read access to the entire volume. A visitor only needs the g (get) permission to view the link.

Volume Flags (volflag):

• dk: Generates per-directory access keys. By default, it only grants access to the folder itself, not subfolders.
• dky: Disables actual key-checking. Anyone with g access can see the folder contents, but subdirectories remain hidden (treated as dotfiles).
• dk + dky: Provides g access where subfolders are hidden.
• dks: Allows users to enter subfolders and enables download-as-zip/tar.

Important Notes:

• If using dirkeys, it is recommended to also enable filekeys (fk) so files within the directory can be hotlinked.
• Dirkeys are generated using --dk-salt + filesystem-path. The key does not change if folder contents change, but it will change if the folder is renamed or the salt is changed.
• Linking to text files for the viewer is not possible if the recipient lacks read-access.

# Example volume flags for directory access
# dk: enable dirkeys
# dky: disable key-check (hide subdirs)
# dks: allow subfolder traversal

IdP volumes are dynamically-created volumes that use ${u} (user) or ${g} (group) in their URL.

Default Behavior (Ephemeral)

By default, IdP volumes are "forgotten" on server restart. They are "revived" only when the owner sends their first request after the restart.

Security Warning: Until a volume is revived, it inherits the permissions of its parent volume.

• If the parent is public, the IdP volume becomes public until revived.
• If the parent is restricted, the IdP volume may become inaccessible until revived.
• Best Practice: Place IdP volumes inside an appropriate parent volume to ensure they inherit safe permissions during the revival window.

Enabling Persistence

Use the idp-store global option to remember users/groups across restarts:

• idp-store: 1 (default): Logs users into a database but does not "remember" them (the setting is effectively ignored).
• idp-store: 2: Remembers usernames only.
• idp-store: 3: Remembers usernames and their groups.

Note: If you use persistence, deleting a user from your external IdP server will not automatically remove them from copyparty. You must manually remove them via the view idp cache option in the control panel.

The up2k protocol is a custom mechanism used by copyparty for efficient, resumable, and parallelized file uploads. It is designed to support non-contiguous chunk uploading directly into the final file, avoiding the slow merge step required by protocols like tus.

Workflow:

1. Chunking: The client splits a file into an "optimal" number of chunks (aiming for $\le 256$ chunks or a total size $\ge 32\text{ MiB}$).
2. Handshake: The client posts a list of chunk hashes, the filename, size, and last-modified timestamp.
3. Wark Creation: The server generates a wark (an identifier for the upload) using sha512(salt + filesize + chunk_hashes) and creates a sparse file.
4. Data Transfer: The client sends a series of POST requests containing one or more consecutive chunks. Each request includes header entries for the chunk-hashes (comma-separated) and the wark ID.
5. Verification: The client performs a final handshake with the hashlist; the server responds with OK or a list of missing/corrupted chunks that need re-uploading.

/* Example of the server log format during an up2k upload */
6.0M 106M/s 2.77G 102.9M/s n948 thank 4/0/3/1 10042/7198 00:01:09