GeeseFS

GeeseFS uses RAM for Metadata (file listings) and Data (read/write buffers).

Data Cache

The default data cache limit is 1 GB (--memory-limit). If multiple processes perform large reads simultaneously, they may exceed this limit.

• To prevent exceeding memory: Use the --use-enomem flag. This will cause GeeseFS to return ENOMEM errors to processes instead of exceeding the limit.
• To allow more memory: Increase --memory-limit (e.g., --memory-limit 4000) or reduce the readahead size using --read-ahead-large (e.g., --read-ahead-large 20).

Metadata Cache

Metadata usage is limited by --entry-limit and --stat-cache-ttl. Non-expired entries cannot be removed from the cache.

When using Yandex S3, you can enable partial object updates (data only) to avoid full server-side copies or re-uploads. This makes fsync cheaper, supports concurrent updates, and improves memory utilization.

To enable this feature, use the --enable-patch flag. Note that new files, metadata changes, and renames still use multipart uploads.

geesefs --enable-patch <bucket> <mountpoint>

The underlying geesefs filesystem uses a non-persistent cache to speed up uploads. If the server shuts down before the cache is flushed, data may be lost without notice.

To guarantee that an object is written to S3 before the upload operation completes, the client must explicitly call fsync. Some clients allow this via configuration. For example, using sftp with the -f flag will perform an fsync after each file upload, ensuring data reaches S3, though this will reduce upload speeds.

sftp -f <server_ip>

GeeseFS does not support concurrent updates of the same file from multiple hosts by default. To avoid lost updates or conflicts, use one of the following strategies:

1. Coordination via fsync(): Ensure one host calls fsync() on the modified file and then waits for at least the duration of --stat-cache-ttl (default is 1 minute) before allowing other hosts to start updating the same file.
2. Forced Cache Invalidation: Use the setfattr command to forcibly refresh the file or directory cache from the server: setfattr -n .invalidate <filename>

If updates are not coordinated, you may see the following warning in the logs: main.WARNING File xxx/yyy is deleted or resized remotely, discarding local changes

GeeseFS can read credentials from the ~/.azure/config file. Ensure the [storage] section is present with account and key defined.

To mount a specific container or a subset of objects under a prefix, use the wasb://container[:prefix] syntax.

# ~/.azure/config format
[storage]
account = "myblobstorage"
key = "MY-STORAGE-KEY"

# Mounting commands
$ $GOPATH/bin/geesefs wasb://container <mountpoint>
$ $GOPATH/bin/geesefs wasb://container:prefix <mountpoint>

To mount an S3 bucket, you can use the geesefs command. GeeseFS automatically looks for credentials in your ~/.aws/credentials file or via AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

Basic Usage

$ geesefs <bucket> <mountpoint>

Mount a specific prefix

If you only want to mount objects under a specific prefix within a bucket:

$ geesefs --endpoint https://... <bucket:prefix> <mountpoint>

Windows Usage

On Windows, the <mountpoint> can be a drive letter (e.g., K:) or a non-existing directory. Credentials can be set via environment variables or a custom config file using --shared-config.

set AWS_ACCESS_KEY_ID=...
set AWS_SECRET_ACCESS_KEY=...
geesefs-win-x64.exe testbucket K:

# Example mounting a bucket to a local directory
geesefs my-bucket /mnt/my-s3-bucket

To prevent passwords from being sent in plain text, it is recommended to enable FTPS.

• Enable Encryption: Set FTP_SSL_ENABLE=YES (optional encryption) or FTP_SSL_ENABLE=FORCE (mandatory encryption).
• Certificates: Place your certificate and key in the /secrets directory as ftp.pem and ftp.key respectively.

If you do not have a certificate, you can generate a self-signed one:

$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout secrets/ftp.key -out secrets/ftp.pem

You can mount Azure Blob Storage using the wasb protocol. GeeseFS supports several authentication methods: using environment variables, an Azure configuration file, or full URIs with the --endpoint flag.

Important Note on Prefixes: If you do not provide a full wasb URI, the prefix separator is a colon (:).

# Using environment variables
$ AZURE_STORAGE_ACCOUNT=xxx AZURE_STORAGE_KEY=yyy \
    $GOPATH/bin/geesefs wasb://container[:prefix] <mountpoint>

To achieve higher linear write speeds (up to ~1.6 GB/s) when you have significant network bandwidth, you should write into multiple files simultaneously and increase parallelism.

Warning: Increasing parallelism with smaller part sizes reduces the maximum supported file size (e.g., using 25 MB parts limits files to 250 GB).

geesefs --no-checksum --memory-limit 4000 \
    --max-flushers 32 --max-parallel-parts 32 --part-sizes 25 <bucket> <mountpoint>

For more explicit configuration, you can provide the full wasb URI containing the account endpoint, or use the --endpoint flag to specify the base URL separately from the wasb URI.

# Using a full wasb URI
$ $GOPATH/bin/geesefs wasb://container@myaccount.blob.core.windows.net/prefix <mountpoint>

# Using the --endpoint flag
$ $GOPATH/bin/geesefs --endpoint https://myaccount.blob.core.windows.net wasb://container:prefix <mountpoint>