MinIO Rust SDK

repository·master·Indexed 18 days ago

https://github.com/minio/minio-rs

A strongly-typed, async-first Rust client for performing bucket and object operations on MinIO or any Amazon S3-compatible object storage. The SDK utilizes a fluent request builder pattern and provides a MinioClient for interacting with S3-compatible services. Version 0.4.0.

Tokens
14.4K
Snippets
42
Records
71
Agent score
60%

What's inside minio-rs

  1. Current limitations and pending features in minio-rs

    master

    This document tracks the feature parity between minio-go and minio-rs. As of the current version, several APIs and features present in the Go SDK are either missing, partially implemented, or require updates in the Rust SDK.

    Key areas of divergence include:

    • Missing APIs: GetObjectAttributes, UpdateObjectEncryption, Inventory APIs, QoS APIs, and LDAP STS provider.
    • Partial Implementations: New AWS checksum algorithms (missing MD5, SHA512, XXHash variants), x-minio-source-time header overrides, and Conditional PUT (requires manual extra_headers injection for now).
    • Infrastructure Gaps: S3 Outposts support and handling of HTTP 200 responses containing XML error bodies (which may currently cause panics in certain response parsing logic).
  2. How the testing strategy is organized

    master

    The SDK uses a multi-layered testing approach to ensure reliability:

    1. Unit Tests: Focused on individual components like type serialization, builder pattern correctness, and response parsing. Located in src/madmin/types/*.rs or via inline #[cfg(test)] modules.
    2. Error Path Tests: Specifically designed to verify error handling (invalid JSON, missing fields, boundary conditions). Located in src/madmin/types/error_tests.rs.
    3. Property-Based Tests: Uses the quickcheck crate to test invariants (like builder idempotence or validation consistency) against arbitrary inputs. Located in src/madmin/builders/property_tests.rs.
    4. Integration Tests: End-to-end workflows with a live MinIO server. Located in the tests/ directory.
  3. Understand the SDK testing architecture

    master

    The MinIO Rust SDK follows an architecture where low unit test coverage (typically ~28%) is expected and normal for an HTTP client library.

    Why unit coverage is low

    Most of the codebase consists of Builders, Clients, and Responses. These components are designed around HTTP request/response cycles and require network I/O or a live server to function. Mocking the entire HTTP stack is considered impractical and provides limited value compared to real integration tests.

    The Testing Strategy

    • Unit Tests: Focus on pure functions, validation logic, encoding/decoding (e.g., url_encode, b64_encode), hashing (sha256_hash), and property-based testing. These are fast and have no external dependencies.
    • Integration Tests: Located in the tests/ directory, these provide the primary confidence by testing end-to-end workflows with a live MinIO server. They cover 100% of the implemented Admin and S3 APIs.

    Mental Model

    Instead of aiming for 100% unit test coverage, the project prioritizes 100% API integration coverage. This ensures that the interaction between the SDK and the actual MinIO server is validated, including real error handling and network cycles.

  4. How the request builder pattern works

    master

    The SDK uses a fluent builder pattern for all S3 operations.

    1. Method Call: Calling a method on MinioClient (e.g., .bucket_exists("name")) returns a specific request builder struct (e.g., BucketExists).
    2. Configuration: You can chain methods on the builder to configure request parameters.
    3. Execution: All request builders implement the S3Api trait, which provides the .send() method. This method is async and returns a typed response (e.g., BucketExistsResponse).

    Internally, builders implement ToS3Request for conversion and responses implement FromS3Response for deserialization.

  5. What to avoid when writing tests

    master

    To maintain a clean and efficient test suite, follow these exclusion rules:

    • Do not test Client Execution Methods: Methods in src/madmin/client/ that call .send() require a live server and should be handled by integration tests, not unit tests.
    • Do not test Trivial Code: Avoid testing simple getters/setters, derived traits (e.g., Debug, Clone), or pass-through wrappers.
    • Do not test External Dependencies: Do not attempt to verify the correctness of reqwest, serde_json, or tokio; assume their behavior is correct.
  6. Debug test failures

    master

    If a test fails and you need more information, you can use the following commands to view detailed output or run a specific test exactly.

    • Use --nocapture to see println! or log output during the test.
    • Use --exact to run only the specific test name provided.
    # View Detailed Output
    cargo test --lib -- --nocapture test_name
    
    # Run Single Test
    cargo test --lib test_name -- --exact
  7. Perform Conditional PUTs via extra_headers

    master

    The typed match_etag and not_match_etag fields for UploadPart, PutObjectContent, and CompleteMultipartUpload are currently pending implementation.

    To perform a conditional PUT (using If-Match or If-None-Match) today, you must manually inject these via the extra_headers field in the respective builders. Note that the SDK does not yet automatically handle the specific quoting or wildcard (*) logic implemented in the Go SDK.

  8. Generate coverage reports for the SDK

    master

    You can use cargo llvm-cov to analyze test coverage. Note that unit test coverage (--lib) does not include coverage from integration tests in the tests/ directory.

    Summary Report

    To see a high-level summary of unit test coverage:

    cargo llvm-cov --lib --summary-only

    HTML Report

    To generate a detailed, line-by-line HTML report:

    cargo llvm-cov --lib --html --output-dir target/coverage

    After running this, open target/coverage/index.html in your browser.

    # Unit test coverage summary
    cargo llvm-cov --lib --summary-only
    
    # HTML report with line-by-line coverage
    cargo llvm-cov --lib --html --output-dir target/coverage
  9. Run micro-benchmarks for the MinIO Rust SDK

    master

    The repository includes low-level micro-benchmarks designed to measure the performance of specific functions or operations. These benchmarks output raw results for performance analysis.

    To run the S3 API benchmarks, use the following command:

    cargo bench --bench s3_api_benchmarks

    Benchmark results are stored in the target/criterion directory.

  10. Guidelines for adding new tests

    master

    When extending the SDK, follow these patterns based on the component type:

    New Type Definitions

    • Add inline serialization tests.
    • Add edge cases to error_tests.rs.
    • Use property tests if the type includes validation logic.

    New Builders

    • Test required parameter validation.
    • Test various combinations of optional parameters.
    • Add property tests for invariants.
    • Verify the resulting request URL, headers, and body.

    New Response Types

    • Test successful parsing using sample JSON.
    • Test error cases such as missing fields or incorrect types.
    • Test handling of optional fields.
  11. Run MinIO Rust SDK examples

    master

    The repository contains several examples for common operations such as uploading, downloading, and prompting objects. You can run them using cargo:

    • Upload a file: cargo run --example file_uploader
    • Upload a file with CLI: cargo run --example put_object
    • Download a file: cargo run --example file_downloader
    • Prompt a file: cargo run --example object_prompt
    cargo run --example <example_name>
  12. Run MinIO Rust SDK tests

    master

    You can run different suites of tests depending on your environment and what you want to verify.

    • All tests: Runs everything, including integration tests (requires a live MinIO server).
    • Unit tests only: Fast execution focusing on library code.
    • Integration tests: Requires a running MinIO server. These are typically ignored by default in standard test runs.
    • Coverage reports: Generates an HTML report to visualize code coverage.
    # All tests
    cargo test
    
    # Unit tests only (Fast)
    cargo test --lib
    
    # Specific test module
    cargo test --lib types::error_tests
    
    # Property-Based Tests
    cargo test --lib property_tests
    
    # Integration Tests (Requires MinIO Server)
    cargo test --test integration_tests -- --ignored
    
    # Coverage Report
    cargo llvm-cov --lib --tests --html --output-dir target/coverage