catfs

Catfs uses a read-ahead and write-through caching semantic.

• Read-ahead: Data is cached as it is accessed.
• Write-through: Data is written to both the source filesystem and the cache.
• Metadata: Currently, all metadata operations hit the source filesystem directly; only data is cached.
• File Caching: If a file is opened for reading, the entire file is cached, even if only a portion is read.
• Non-sequential Writes: If the filesystem (like goofys) emits ENOTSUP for non-sequential writes, catfs falls back to flushing the entire file on close(). This means changing a single byte may trigger a full rewrite of the file.

Warning: Catfs is ALPHA software. Do not use it if you value your data.

You can install catfs on Linux using pre-built binaries from the GitHub releases page (ensure fuse-utils is installed) or by building from source using Cargo.

$ cargo install catfs
# The binary will be located in $HOME/.cargo/bin/catfs

Catfs provides cached access to a source filesystem by exposing its contents at a mountpoint and caching data to a target directory.

Requirements:

• The target filesystem (<to>) must have extended attributes (xattr) enabled (typically via the user_xattr mount option).

Command Syntax: catfs <from> <to> <mountpoint>

• <from>: The source filesystem (e.g., a remote directory or local path).
• <to>: The directory where data will be cached.
• <mountpoint>: The location where the cached filesystem will be mounted.

Options:

• --free: Controls how much free space the <to> filesystem must maintain.

$ catfs /path/to/source /path/to/cache /path/to/mountpoint

The PCatFS struct is designed to bridge the synchronous FUSE Filesystem trait with a multi-threaded execution model.

• Concurrency: It uses a ThreadPool with a fixed size of 100 threads.
• Lifecycle: When PCatFS is dropped, it calls self.tp.join(), ensuring all pending filesystem operations in the thread pool are completed before the object is destroyed.
• Asynchronous Execution: Most filesystem operations (like getattr, lookup, read, mkdir, etc.) are offloaded to the thread pool using the run_in_threadpool! macro. This allows the FUSE driver to handle requests without waiting for the underlying disk/cache I/O to complete synchronously.

To automatically mount catfs at boot, add an entry to /etc/fstab. The entry follows the format: catfs#<source>#<cache> <mountpoint> fuse <options> 0 0.

Example entry:

catfs#/src/dir#/cache/dir /mnt/point    fuse    allow_other,--uid=1001,--gid=1001,--free=1%   0       0

Catfs uses env_logger. You can control the verbosity of the output by setting the RUST_LOG environment variable. If not set, it defaults to info level.

When running in the background (daemon mode), logs are sent to syslog (Facility: LOG_USER). When running in the foreground, logs are printed to standard output.

RUST_LOG=debug catfs /src /cache /mnt

To run the official benchmarks, use the kahing/catfs-bench Docker image. The benchmark results are written to $PWD/target inside the container, which is mapped to your host directory.

Basic Benchmark Command:

$ sudo docker run -e SSHFS_SERVER=user@host --rm --privileged --net=host -v $PWD/target:/root/catfs/target kahing/catfs-bench

Benchmark with SSH Socket Mounting: To allow the container to use your host's SSH configuration (e.g., for sshfs testing), mount your host's SSH sockets:

$ sudo docker run -e SSHFS_OPTS="-o ControlPath=/root/.ssh/sockets/%r@%h_%p -o ControlMaster=auto -o StrictHostKeyChecking=no -o Cipher=arcfour user@host:/tmp" -e SSHFS_SERVER=user@host --rm --privileged --net=host -v $HOME/.ssh/sockets:/root/.ssh/sockets -v $PWD/target:/root/catfs/target kahing/catfs-bench

Use Handle::open to access an existing file. This method checks if a valid cache exists. If the cache is valid, it uses it; otherwise, it synchronizes the source file to the cache.

Parameters:

• src_dir: RawFd for the source directory.
• cache_dir: RawFd for the cache directory.
• path: Path to the file.
• flags: File opening flags.
• cache_valid_if_present: If true, the cache is considered valid if it exists, even without a checksum check.
• disable_splice: If true, uses user-space copying instead of the splice system call for performance/compatibility.
• tp: A Mutex<ThreadPool> used for background read-ahead (page-in) operations.

let handle = Handle::open(
    src_dir_fd,
    cache_dir_fd,
    Path::new("existing.txt"),
    libc::O_RDWR,
    true,
    false,
    &thread_pool_mutex,
)?;

Catfs uses a custom error wrapper called RError<E> which attaches a backtrace to errors. For most standard operations, you should use the Result<T> type alias, which is specialized as std::result::Result<T, RError<io::Error>>.

Key features of RError:

• Backtrace Support: When created via RError::from(e), it captures a backtrace. It includes logic to trim the backtrace to exclude internal error-handling frames (specifically those from error.rs).
• Deref Implementation: RError<E> implements Deref, allowing you to access the underlying error E directly.
• Display: When printing an RError<io::Error>, it includes both the error message and the captured backtrace if available.

use catfs::error::Result;

fn do_something() -> Result<()> {
    // ... logic that might return an io::Error
    Ok(())
}

CatFS uses the extended attribute user.catfs.src_chksum to track if a cache file is 'pristine' (perfectly synchronized with the source).

• set_pristine(true): Calculates a SHA512 checksum of the source file (based on its s3.etag xattr, mtime, and size) and stores it in the cache file's user.catfs.src_chksum attribute.
• set_pristine(false): Removes the user.catfs.src_chksum attribute from the cache file, marking it as potentially out-of-sync.

// Mark the current cache as synchronized with source
handle.set_pristine(true)?;

// Mark the cache as dirty/invalid
handle.set_pristine(false)?;