parsync Documentation

You can install parsync using several methods depending on your operating system:

Linux and macOS

Use the install script via curl:

curl -fsSL https://alpindale.net/install.sh | bash

Windows

Use PowerShell:

powershell -ExecutionPolicy Bypass -c "irm https://alpindale.net/install.ps1 | iex"

Rust/Cargo

If you have Rust installed, use cargo:

cargo install parsync

From Source

Clone the repository and use make:

make build
make install

Use parsync to perform high-throughput, resumable transfers between SSH remotes and local paths, or between local directories.

Basic Syntax

parsync [options] <source> <destination>

SSH Remote Syntax

To sync from a remote host, use the user@host:/path format. If the remote host uses a non-default SSH port, specify it as user@host:port:/path.

Supported Features

• Parallel file transfers.
• Optional block-delta sync.
• SSH config host aliases are supported.

parsync -vrPlu user@example.com:/remote/path /local/destination

Parsync handles system interrupts (like Ctrl+C) using a two-stage approach:

1. First Interrupt: The application catches the signal, sets an INTERRUPTED flag, and prints a message: [parsync] interrupt received, stopping after current operation.... The current file operation is allowed to finish to maintain state integrity.
2. Second Interrupt: If a second interrupt is received while waiting for the first to finish, the process forces an immediate exit with exit code 130.

To handle multiple concurrent workers, use ConnectionPool. It manages a set of reusable Connection objects to avoid the overhead of repeated SSH handshakes.

Workflow:

1. Initialize the pool with ConnectionPool::new(target, pool_size). This eagerly opens one connection to validate authentication.
2. Use pool.checkout() to acquire a PooledConnection. This will either return an idle connection or create a new one if the max_size hasn't been reached. If the pool is full, it will block until a connection is returned.
3. The PooledConnection implements Drop, so when it goes out of scope, the connection is automatically returned to the pool via checkin rather than being closed.

let pool = ConnectionPool::new(target, 4)?;
{
    let mut conn = pool.checkout()?;
    conn.exec("whoami")?;
}
// conn is automatically checked back into the pool here

To prevent directory traversal attacks and ensure sync integrity, parsync validates destination paths using validate_destination_path.

Security Rules:

• No Absolute Paths: Remote entries must not contain absolute paths.
• No Traversal: Path components like .. (ParentDir), / (RootDir), or platform-specific prefixes are forbidden.
• Symlink Protection: The destination path must not traverse a symlink that exists outside of the designated sync root.
• Directory Integrity: If a path component is expected to be a directory (e.g., for a file inside a folder), the system verifies that the component actually exists as a directory on the local filesystem.

parsync resolves configuration settings using a specific hierarchy of precedence. When a setting is defined in multiple places, the one higher in this list takes priority:

1. CLI Flags: Arguments passed directly to the parsync command.
2. Environment Variables: Variables prefixed with PARSYNC_.
3. Configuration File: Settings defined in the TOML configuration file.
4. Defaults: Built-in default values used if no other configuration is provided.

This allows you to set global defaults in a config file, override them for specific sessions using environment variables, or apply one-off overrides via CLI flags.

Parsync supports high-performance transfer modes like RDMA and fast-copy.

RDMA (Remote Direct Memory Access):

• Uses a specialized fast path for data transfer.
• If strict_durability is enabled, the partial file is synced to disk (sync_all) before verification.
• If the remote file's size or modification time changes during the RDMA transfer, the system will either fail (if RdmaMode::Require is set) or fall back to a full transfer.

Fast-copy:

• A secondary fast path that uses remote.try_fast_copy to move data into a partial file.
• Like RDMA, it supports strict_durability via sync_all on the partial file.
• If the fast-copy fails, the partial file is removed.

The Connection logic attempts authentication in the following order:

1. SSH Agent: Checks if the user is already authenticated via an active SSH agent.
2. Public Key Files: Tries configured identity files, then defaults to ~/.ssh/id_ed25519 and ~/.ssh/id_rsa.
3. Environment Variable: If the above fail, it attempts to use the password provided in the PARSYNC_SSH_PASSWORD environment variable.

If all methods fail, authentication will error.

When syncing files, you can preserve various metadata attributes from the remote source to the local destination.

Supported attributes include:

• Permissions: Uses preserve_perms to set file modes.
• Owner/Group: Uses preserve_owner and preserve_group to set UID/GID (Unix).
• Extended Attributes (xattrs): Uses preserve_xattrs to sync key-value pairs.
• ACLs: Uses preserve_acls to sync Access Control Lists (Unix via setfacl).

Platform Notes:

• Unix: Full support for permissions, owner, group, xattrs, and ACLs.
• Windows: Support for permissions is limited (sets readonly based on mode). Owner, group, xattrs, and ACLs are generally unsupported. If strict_windows_metadata is enabled, attempting to preserve these unsupported attributes will cause the sync to fail with an error.