S3Mock

S3Mock is a lightweight server that implements a subset of the Amazon S3 API for local integration testing. It is designed to replace real AWS infrastructure during development and testing cycles.

Recommended usage: To avoid classpath conflicts, it is recommended to run S3Mock as a Docker container or via Testcontainers.

S3Mock provides experimental support for the Amazon S3 Vectors API for storing and querying high-dimensional vector embeddings. To enable this, you must activate the vectors Spring profile. The Vectors API runs on dedicated ports separate from standard S3 ports: 9092 (HTTP) and 9193 (HTTPS).

Supported Operations:

• Create/Get/List/Delete VectorBuckets
• Put/Get/List/Delete VectorBucketPolicies
• Create/Get/List/Delete Indexes
• Put/Get/List/Delete Vectors
• QueryVectors
• Tag/Untag/ListResource Tags

# Enable vectors via Docker
docker run -p 9090:9090 -p 9191:9191 -p 9092:9092 -p 9193:9193 \
  -e SPRING_PROFILES_ACTIVE=vectors \
  adobe/s3mock

S3Mock adheres to specific API and URL constraints that users must be aware of when integrating it into their workflows:

• URL Style: S3Mock only supports path-style URLs (e.g., http://localhost:9090/<bucket>/<key>). Virtual-hosted-style URLs (e.g., bucket.localhost) are not supported.
• XML Compliance: XML element and attribute names match the AWS S3 API specification exactly.
• Presigned URLs: S3Mock accepts presigned URLs, but it does not validate the signatures. Do not rely on signature validation for security testing within S3Mock.

S3Mock has no security by design. It is strictly a local testing tool and should never be used to store real credentials or PII.

Key security behaviors:

• Authentication/Authorization: None. All requests are accepted regardless of credentials or AWS Signature V4. Any client can read or write any bucket or object.
• Presigned URLs: Accepted, but signatures are parsed and not validated.
• KMS: The KmsValidationFilter validates the format of KMS ARNs, but no actual encryption or key material is used.
• SSL/TLS: Uses a self-signed certificate (s3mock.jks). Clients must disable certificate validation when connecting via HTTPS.

S3Mock uses Spring Boot profile groups to compose behavior. Actuator endpoints are disabled by default (management.endpoints.access.default=none).

Profile groups are defined in application.properties:

spring.profiles.group.debug=actuator
spring.profiles.group.trace=actuator

Specific properties are managed in:

• application-debug.properties: Logging levels.
• application-trace.properties: Logging levels.
• application-actuator.properties: JMX and actuator endpoint settings.

S3Mock uses a filesystem-based storage model where data is split into binary bodies and JSON metadata sidecars.

Storage Structure

• Binary Data: Stored as a flat file at <root>/<bucket>/<uuid>/binaryData.
• Metadata: Stored as a JSON file at <root>/<bucket>/<uuid>/objectMetadata.json.
• Bucket Mapping: The BucketStore maintains a key to UUID mapping in a bucketMetadata.json file.

Concurrency and Locking

S3Mock uses a ConcurrentHashMap lock store to manage concurrency. There are no cross-store transactions.

• Bucket-level: All writes to the same bucket are serialized through a per-bucket synchronized lock.
• Object-level: All writes to the same object are serialized through a per-object synchronized lock.

To maintain code quality, use the following modern Java patterns:

Local Type Inference

Use var for local variables when the type is clear from the context. Avoid it if the type is ambiguous.

var uploadFile = new File(UPLOAD_FILE_NAME);
var response = s3Client.getObject(...);

Collections

• Use list.isEmpty() or !list.isEmpty() instead of checking list.size() == 0.
• Use List.of(...) and Map.of(...) for immutable collections instead of Collections.unmodifiableList(...).
• Prefer Streams over explicit loops for transformations:

buckets.stream().map(Bucket::name).collect(Collectors.toSet())

Switch Expressions and Text Blocks

• Prefer switch expressions over if-else chains when dealing with 3 or more branches.
• Use text blocks for all multi-line strings.

Keep the following limitations in mind when using S3Mock:

• Path-style access only: Use http://localhost:9090/bucket/key. Virtual-hosted style (http://bucket.localhost:9090/key) is not supported.
• Presigned URLs: Accepted but not validated (expiration, signature, and HTTP verb are not checked).
• Self-signed SSL: Clients must trust the bundled certificate or disable SSL verification.
• KMS: Key ARNs are validated, but no actual encryption is performed.
• Not for production: S3Mock is a testing tool and lacks production security features.

S3Mock persists data to the local filesystem using the following hierarchy. Note that this structure is an implementation detail and may change; reusing persisted data across restarts is not officially supported.

<root>/
  <bucket-name>/
    bucketMetadata.json              # Bucket metadata
    <object-uuid>/
      binaryData                     # Object content
      objectMetadata.json            # Object metadata
      <version-id>-binaryData        # Versioned object (if versioning enabled)
      <version-id>-objectMetadata.json
    multiparts/
      <upload-id>/
        multipartMetadata.json
        <part-number>.part
        <part-number>.partmeta.json # per-part checksum + size; removed after CompleteMultipartUpload

S3Mock is designed strictly as a local testing tool and is not suitable for production use. Its security model has the following characteristics:

• No Auth: It has no authentication or authorization mechanisms.
• No Encryption: It performs no real encryption. While KMS key ARN formats are validated, no actual encryption occurs.
• Self-signed SSL: The SSL certificate is self-signed and intended only for local testing environments.

S3Mock is designed as a local testing tool and does not implement authentication or authorization.

Security Model:

• No Validation: S3Mock accepts all requests regardless of credentials. AWS Signature V4 headers are parsed for compatibility but never validated. Presigned URLs are accepted but signatures are not checked.
• Access Control: Any client can read or write any bucket or object.
• Encryption: KMS key ARN formats are validated to catch configuration errors, but no actual encryption is performed.
• SSL/TLS: The SSL certificate is self-signed; clients must be configured to disable certificate validation.

WARNING: Never deploy S3Mock in a shared or internet-accessible environment.