Mountpoint for Amazon S3

The mountpoint-s3-fs crate implements the core functionality of Mountpoint for Amazon S3, which is a high-throughput file client designed to mount an Amazon S3 bucket as a local file system.

WARNING: This library is currently unstable. The API interface is subject to significant changes and is not intended for general-purpose use. For production or stable environments, use the official Mountpoint releases instead.

The mountpoint-s3-crt crate provides a specialized Rust interface to the AWS Common Runtime. It is designed specifically for use by Mountpoint for Amazon S3 and includes only the subset of AWS Common Runtime features required by Mountpoint.

Warning: This crate is not intended for general-purpose use and its interface is considered unstable. For general-purpose AWS client functionality in Rust, use the official AWS SDK for Rust.

The mountpoint-s3-client crate is a high-performance Amazon S3 client specifically designed for use by Mountpoint for Amazon S3. It binds to the AWS Common Runtime (CRT) to provide AWS authentication, an HTTP client, and low-level IO primitives.

Warning: This crate is not intended for general-purpose use and its interface is considered unstable. For general-purpose Amazon S3 client needs in Rust, use the official AWS SDK for Rust instead.

Mountpoint for Amazon S3 is a Linux FUSE file system that translates file operations (like open and read) into S3 object API calls. It is organized as a Rust workspace with a layered dependency hierarchy:

• mountpoint-s3: The main binary crate providing the CLI and mount logic.
• mountpoint-s3-fs: The core filesystem implementation, including caching, prefetching, and inode management.
• mountpoint-s3-client: An SDK-like S3 client that constructs requests (e.g., HeadObject, DeleteObject) and provides streaming abstractions.
• mountpoint-s3-fuser: A fork of the fuser crate providing low-level FUSE protocol handling and kernel interfaces.
• mountpoint-s3-crt: High-level, idiomatic Rust bindings for the AWS Common Runtime.
• mountpoint-s3-crt-sys: Low-level system bindings for the AWS Common Runtime.

Mountpoint for Amazon S3 is a high-throughput file client optimized for specific access patterns. It is not a full POSIX-compliant file system.

Recommended Use Cases

• Reading large objects from S3 (potentially concurrently from many instances) without local downloading.
• Accessing unpredictable subsets of S3 objects.
• Uploading output to S3 directly or using tools like cp to upload local files.

Unsupported Operations and Limitations

• Non-supported POSIX features: Directory renaming and symlinks are not supported.
• File Edits: Do not perform in-place edits to existing files (e.g., using vim or working directly on a Git repository within the mount).
• Compatibility: While it may work with other S3-compatible storage services, it is explicitly optimized for Amazon S3 and support for other services is not guaranteed.

Mountpoint for Amazon S3 provides a file system interface to S3 objects, optimized for high read throughput of large objects and sequential writes of new objects from a single client.

Key Limitations:

• It does not implement all POSIX features (e.g., no rename on general purpose buckets, no mutable ownership/permissions, no hard or symbolic links).
• It fails early with IO errors rather than silently accepting unsupported operations (like extended attributes).
• If your application requires rich POSIX semantics, consider Amazon EFS or Amazon FSx instead.

Mountpoint interprets S3 object keys as file system paths by splitting them on the / character.

Key Restrictions:

• Null bytes: Keys containing \0 are inaccessible.
• Reserved names: Keys resulting in . or .. (including /., /.., /./, or /../) are inaccessible as these are reserved for POSIX relative directories.
• Trailing delimiters: Keys ending in / are treated as directories. The object itself is not accessible, but a directory of that name will be visible.
• Shadowing: Files are shadowed by directories with the same name. If you have keys blue (file) and blue/image.jpg (file), the blue directory will take precedence, making the blue file inaccessible.
• Path Delimiters: Windows-style backslashes (\) are not supported.

Mountpoint for Amazon S3 uses fio to measure performance across several workload types:

• Read Workload: Measures throughput (with and without caching) and latency (time to first byte). Common variants include:
  • four_threads: Concurrent execution using four fio threads.
  • direct_io: Bypassing kernel page cache using O_DIRECT (Linux only).
  • small_file: Testing against 5 MiB files instead of the standard 100 GiB.
• Readdir Workload: Measures the time to run ls against directories containing between 100 and 100,000 files.
• Write Workload: Measures write throughput using sequential write simulations.

Mountpoint supports general purpose buckets, directory buckets (S3 Express One Zone), and Outposts buckets. Use the appropriate naming convention for each type:

• General Purpose: Use the full bucket name.
• Directory Buckets: Use the full bucket name including the --az_id--x-s3 suffix.
• Outposts Buckets: Use the access point ARN or alias.

On Amazon Linux 2023 (version 2023.9.20251110 or later), Mountpoint is available directly in the official repository.

1. Install using dnf:

   sudo dnf install mount-s3

2. Verify the installation:

   mount-s3 --version

To publish Mountpoint metrics directly via Prometheus, use Prometheus v3.0 or later. You must start Prometheus with specific feature flags to support the OTLP exponential histograms and delta temporality used by Mountpoint. Without these flags, histogram data may be dropped or misinterpreted.

prometheus \
  --config.file=prometheus.yml \
  --web.listen-address=:9090 \
  --web.enable-otlp-receiver \
  --enable-feature=native-histograms,otlp-deltatocumulative