s3mini Documentation

repository·dev·Indexed 23 days ago

https://github.com/good-lly/s3mini

An ultra-lightweight, zero-dependency TypeScript client for S3-compatible object storage, optimized for Node.js, Bun, and edge platforms like Cloudflare Workers. It provides a compact (~20 KB minified) implementation for common S3 operations including smart uploads via putAnyObject, multipart uploads, object listing, and server-side encryption (SSE-C). Version 1.0.0 is ESM-only and includes native fast paths for Bun.

Tokens
16.4K
Snippets
42
Records
68
Agent score
79%

What's inside s3mini

  1. Upload objects with putAnyObject (Smart Upload)

    dev

    The putAnyObject method is the recommended way to upload data. It automatically decides between a single PUT request and a multipart upload based on the data size.

    • ≤ 8MB (default): Performs a single PUT request.
    • > 8MB: Automatically performs a multipart upload with 4 concurrent part uploads, automatic retries (3 attempts with exponential backoff), and automatic cleanup on failure.

    Memory Efficiency Tip: For large files, use Blob or File instead of Uint8Array to enable zero-copy slicing, which prevents loading the entire file into memory.

    // Small file — uses single PUT internally
    await s3.putAnyObject('small.txt', 'Hello World');
    
    // Large file — automatically uses multipart
    const largeBuffer = await fs.readFile('video.mp4'); // 500MB
    await s3.putAnyObject('videos/movie.mp4', largeBuffer, 'video/mp4');
    
    // Blob (zero-copy slicing for memory efficiency)
    const file = new File([largeArrayBuffer], 'data.bin');
    await s3.putAnyObject('uploads/data.bin', file);
    
    // ReadableStream (uploads as data arrives)
    const stream = fs.createReadStream('huge-file.dat');
    await s3.putAnyObject('backups/data.dat', Readable.toWeb(stream));
  2. Configure s3mini environment variables

    dev

    To use s3mini, you must provide provider credentials and the S3 endpoint via environment variables. It is recommended to create a .env file in your project root. You can use the provided example.env as a template.

    # On Windows, Mac, or Linux
    mv example.env .env
  3. Use S3mini in Cloudflare Workers

    dev

    s3mini works natively in Cloudflare Workers without requiring nodejs_compat compatibility mode. You can pass environment variables directly into the constructor.

    export default {
      async fetch(request: Request, env: Env): Promise<Response> {
        const s3 = new S3mini({
          accessKeyId: env.R2_ACCESS_KEY,
          secretAccessKey: env.R2_SECRET_KEY,
          endpoint: env.R2_ENDPOINT,
        });
    
        const data = await s3.getObject('hello.txt');
        return new Response(data);
      },
    };
  4. Install s3mini via npm, yarn, or pnpm

    dev

    You can install s3mini using your preferred package manager. This library is an ultra-lightweight TypeScript client (~20 KB minified) designed for Node.js, Bun, and Cloudflare Workers. Note that it does not support browser environments.

    npm install s3mini
    
    yarn add s3mini
    
    pnpm add s3mini
  5. Quick Start with s3mini

    dev

    To get started, instantiate S3mini with your credentials and endpoint, then use the provided methods for common S3 operations like uploading, downloading, listing, and deleting objects.

    import { S3mini } from 's3mini';
    
    const s3 = new S3mini({
      accessKeyId: process.env.S3_ACCESS_KEY,
      secretAccessKey: process.env.S3_SECRET_KEY,
      endpoint: 'https://bucket.region.r2.cloudflarestorage.com',
      region: 'auto',
    });
    
    // Upload (auto-selects single PUT or multipart based on size)
    await s3.putAnyObject('photos/vacation.jpg', fileBuffer, 'image/jpeg');
    
    // Download
    const data = await s3.getObject('photos/vacation.jpg');
    
    // List
    const objects = await s3.listObjects('/', 'photos/');
    
    // Delete
    await s3.deleteObject('photos/vacation.jpg');
  6. Handle Directory Prefixes in listObjects (v0.8.1)

    dev

    Starting from version 0.8.1, listObjects and listObjectsPaged include CommonPrefixes in results when the delimiter option is used.

    Directory prefixes are returned as synthetic ListObject entries with the following properties:

    • Key: Ends in / (e.g., prefix/subdir/)
    • Size: 0
    • ETag: ''
    • StorageClass: ''
    • LastModified: new Date(0)

    Migration: If your logic expects only files, you must now filter the results by checking if the Key ends with a slash or by checking for a non-empty ETag.

    // Before: only files returned
    const objects = await s3.listObjects('/', 'prefix/', undefined, { delimiter: '/' });
    
    // After: files + directory prefixes returned
    const all = await s3.listObjects('/', 'prefix/', undefined, { delimiter: '/' });
    const files = all.filter(o => !o.Key.endsWith('/'));
    const directories = all.filter(o => o.Key.endsWith('/'));
  7. Migrate to s3mini v1.0.0 (ESM and Bun Fast Paths)

    dev

    Version 1.0.0 introduced several significant changes:

    ESM Only

    s3mini is now an ESM-only package. The minified CJS build has been removed.

    Bun Native Fast Paths

    On Bun, several methods now use Bun.S3Client instead of globalThis.fetch. This improves performance but changes how requests are handled:

    • Affected Methods: getObject, getObjectArrayBuffer, getObjectJSON, getEtag, getContentLength, objectExists, deleteObject, listObjects, and getPresignedUrl.
    • Behavioral Changes: These requests bypass globalThis.fetch, meaning fetch mocks, interceptors, requestAbortTimeout, and per-request logger output will not work for them.
    • Error Handling: Errors are still S3ServiceError, but .status may be 0 for certain codes, and .body contains the provider's message text instead of raw XML.
    • Presigned URLs: Uses Bun's signer; query parameters may differ from other runtimes.

    Migration: If you need to maintain the previous fetch-based behavior on Bun, pass a custom fetch implementation in your configuration:

    const s3 = new S3mini({ ...config, fetch: (input, init) => fetch(input, init) });

    Upload Methods

    putObject, putAnyObject, getObjectRaw, and getObjectWithETag no longer use Bun fast paths to avoid issues with Bun.write() rewriting content types or mishandling ReadableStream objects. These methods now use the standard signed path on all runtimes.

  8. Perform manual multipart uploads

    dev

    For advanced scenarios requiring progress tracking, resumable uploads, or custom concurrency, use the manual multipart upload workflow:

    1. Initialize: Call getMultipartUploadId to get an uploadId.
    2. Upload Parts: Use uploadPart to upload chunks. Each part must be $\ge$ 5MB (except the last part). Part numbers are 1-indexed (max 10,000).
    3. Complete: Call completeMultipartUpload with the array of completed parts.
    4. Abort/Cleanup: Use abortMultipartUpload to cancel an upload or listMultipartUploads to find and clean up orphaned uploads.
    // 1. Initialize upload
    const uploadId = await s3.getMultipartUploadId(
      key: string,
      contentType?: string,
      ssecHeaders?: SSECHeaders,
      additionalHeaders?: AWSHeaders,
    );
    
    // 2. Upload parts (must be ≥ 5MB except last part)
    const parts: UploadPart[] = [];
    
    for (let i = 0; i < totalParts; i++) {
      const partData = buffer.subarray(i * partSize, (i + 1) * partSize);
      const part = await s3.uploadPart(
        key,
        uploadId,
        partData,
        i + 1,  // partNumber: 1-indexed, max 10,000
      );
      parts.push(part);
      console.log(`Uploaded part ${i + 1}/${totalParts}`);
    }
    
    // 3. Complete upload
    const result = await s3.completeMultipartUpload(key, uploadId, parts);
    console.log('Final ETag:', result.etag);
  9. Use DataInput for uploading data

    dev

    The DataInput type defines the various formats accepted for object data. You can upload data as a string, ArrayBuffer, Uint8Array, ReadableStream, File, or Blob.

    type BinaryData = ArrayBuffer | Uint8Array;
    
    type MaybeBuffer = typeof globalThis extends { Buffer?: infer B }
      ? B extends new (...a: unknown[]) => unknown
        ? InstanceType<B> | BinaryData
        : BinaryData
      : BinaryData;
    
    export type DataInput = string | MaybeBuffer | ReadableStream | File | Blob;