DOCSEARCHbyAlgolia
Submit Docs

Apache OpenDAL Reqsign

repository·main·Indexed 41 days ago

https://github.com/apache/opendal-reqsign

A Rust library designed to simplify signing API requests for cloud storage services, such as AWS SigV4 and Azure Storage. It utilizes a 'build, sign, send' workflow and provides a modular system of traits for credential loading and request signing, with support for Tokio-based file reading and reqwest HTTP transport.

Tokens
98.9K
Snippets
278
Records
377
Agent score
51%

What's inside OpenDAL Reqsign

  1. Use reqsign-aws-core for AWS signing

    main

    The reqsign-aws-core crate provides the foundational AWS credential types, algorithm-independent credential providers, and canonical request primitives used by SigV4 and SigV4a signing implementations.

    Note for most users: You should generally not depend on reqsign-aws-core directly. Instead, depend on reqsign-aws-v4 or reqsign-aws-v4a, which re-export the necessary public types for your signing needs.

  2. Understand Reqsign's security boundary and guarantees

    main

    Reqsign's security boundary encompasses the public library boundary and the internal state it manages. When using the library, you can rely on the following security guarantees:

    • Memory Safety & API Integrity: Public APIs like Signer::sign, SigningRequest::build, and SigningRequest::apply are designed to prevent memory-safety violations, data races, and panics. They are guaranteed to preserve request properties (method, scheme, authority, path, query, and headers) according to their contract.
    • Credential Isolation: Credentials are isolated between independent signers, providers, and contexts. A request signed via Signer B will not use credentials from Signer A unless explicitly shared by the caller.
    • Credential Redaction: Secrets (access keys, private keys, tokens, etc.) are redacted from Debug output, error chains, logs, and panic messages. Errors may identify the provider or error type but will not include the secret values themselves.
    • Source Control: Credential providers only access the sources explicitly configured or documented. For example, a Context::new() does not silently read environment variables or local files; it uses no-op adapters unless configured otherwise.
    • Request Correctness: Signers (such as AWS SigV4, Azure, or Google) are responsible for canonicalizing requests (method, path, headers, etc.) according to their specific service contracts.
    • WASM Safety: In wasm32-unknown-unknown builds, Reqsign ensures that no accidental OS environment, filesystem, or command execution access is enabled through no-op context components.

  3. Use caller-provided subject tokens for cloud identity exchange

    main

    If your application already possesses an OIDC or other external assertion (subject token), you can bind it directly to an AWS, Azure, or Google credential provider.

    Important Security Note: The host application is responsible for authenticating and authorizing the incoming identity before passing the token to reqsign. reqsign only performs the cloud exchange and request signing. Subject tokens are opaque to reqsign and are never added to a default credential chain.

    Expiration Handling: If you provide an absolute expiration for the subject token, reqsign checks it before performing exchange I/O. If no expiration is provided, the token remains subject to the authoritative service's validation.

  4. Understand the Reqsign Security Boundary and Responsibility Model

    main

    Reqsign operates under a specific security model where it assumes the caller (host application) is a trusted entity. To use Reqsign securely, developers must understand what the library protects and what responsibilities fall on the application integrating it.

    What Reqsign Protects (In-Scope)

    • Credential Isolation: Ensuring one signer does not use credentials from another.
    • Public API Safety: Maintaining memory safety and predictable behavior through documented APIs.
    • Request Integrity: Ensuring the signing process does not inadvertently leak secrets or sign incorrect bytes due to internal logic errors.
    • Explicit Capabilities: Context::new() and Context::default() are no-op by default; ambient access to files, network, or environment variables is only granted through explicit adapters.

    What the Caller Must Handle (Out-of-Scope)

    • Authentication & Authorization: Reqsign does not decide if an end-user is allowed to perform an action or if a specific URL/header is safe to sign.
    • Endpoint & Provider Selection: The caller is responsible for choosing the correct service names, regions, accounts, and HTTP endpoints. Signing a request for an attacker-controlled endpoint is a caller configuration issue, not a Reqsign vulnerability.
    • Transport Security: TLS validation, CA bundles, proxies, and DNS are managed by the configured HTTP client and host environment, not Reqsign.
    • Cloud Provider Policy: Reqsign does not validate if an IAM policy or bucket policy is overprivileged; it only ensures the signature matches the provider's rules.
    • End-to-End Integrity: Reqsign signs requests, but it does not provide integrity guarantees for the response bytes or backend state.

  5. Configure the Default Credential Chain

    main

    The DefaultCredentialProvider::new() follows a specific search order to find credentials:

    1. env
    2. azure_cli (non-wasm32 only)
    3. client_certificate (non-wasm32 only)
    4. client_secret
    5. azure_pipelines
    6. workload_identity
    7. imds

    You can customize which slots are used by using the DefaultCredentialProvider::builder().

    use reqsign_azure_storage::{
    ClientSecretCredentialProvider, DefaultCredentialProvider,
};

let provider = DefaultCredentialProvider::builder()
    .no_env()
    .client_secret(
        ClientSecretCredentialProvider::new().with_tenant_id("tenant-id"),
    )
    .no_imds()
    .build();

  6. How reqsign-core abstractions work together

    main

    The reqsign-core crate uses several key abstractions to orchestrate the signing process:

    1. Context: A container for runtime dependencies. It provides access to the file system (via FileRead), HTTP clients (via HttpSend), and environment variables (via Env).
    2. Signer: The central orchestrator. It follows a three-step lifecycle:
      • Loads credentials using a ProvideCredential implementation.
      • Builds service-specific signing requests using a SignRequest implementation.
      • Applies the resulting signatures to the HTTP request parts.
    3. Traits: The system is extended by implementing specific traits:
      • ProvideCredential: Defines how to load credentials from various sources.
      • SignRequest: Defines how to construct the signing request for a specific service.
      • SigningCredential: Used to validate the validity of credentials.
      • FileRead, HttpSend, and Env: Provide the underlying capabilities for the Context to interact with the system and network.

  7. Identify protected assets in Reqsign

    main

    Reqsign is designed to protect several specific types of sensitive data and configurations during the signing process:

    • Credentials & Tokens: Signing credentials, temporary tokens, private keys, shared keys, bearer tokens, and reusable derived signing material.
    • Request Metadata: The specific request fields covered by the signing contract (e.g., method, scheme, authority, path, query, signed headers, payload hash, expiry, service, region, and account values).
    • Configuration State: Explicit caller configurations for credential sources, signing methods, service, region, expiry, and context capabilities.
    • Isolation: Maintaining credential isolation between independent signing setups (unless explicitly shared by the caller).

  8. Configure Google Cloud request signing

    main

    When using RequestSigner, you can specify the target Google service (e.g., "storage").

    For query signing, the signer can use a specific identity via IAMCredentials signBlob. The identity selection follows this precedence:

    1. An email explicitly set using RequestSigner::with_signer_email.
    2. An identity discovered by the credential provider via impersonation or VM metadata configuration.

    Additionally, you can enable client-side token generation for Cloud Storage Credential Access Boundary downscoping by enabling the credential-access-boundary-client-side feature.

  9. How DefaultCredentialProvider slots work

    main

    The DefaultCredentialProvider uses a slot-based model to manage the credential chain. Each slot represents a specific credential source (e.g., env, profile, imds).

    Internal Logic

    • Enabled with Default: When you use DefaultCredentialProvider::builder(), all documented default slots are initialized to Some(T::default()).
    • Enabled with Custom: Using .slot_name(provider) sets the slot to Some(custom_provider).
    • Disabled: Using .no_slot_name() sets the slot to None.

    When .build() is called, the resulting chain only includes the slots that are Some(...). This ensures that the credential search order is predictable and only includes the sources you have explicitly enabled or left as defaults.

  10. Configure Volcengine TOS authentication methods

    main

    The reqsign-volcengine-tos implementation supports two authentication modes:

    1. Header authentication
    2. Presigned query authentication

    Credentials can be managed via the DefaultCredentialProvider (reading from environment variables) or a StaticCredentialProvider (for credentials supplied directly by your application).

  11. Understand the security boundary between Reqsign and host applications

    main

    Reqsign is a reusable library designed to be embedded in host applications like Apache OpenDAL. When using Reqsign, it is critical to understand the division of security responsibilities to ensure the overall system is secure.

    Reqsign Responsibilities

    Reqsign is responsible for the following internal security properties:

    • Credential Loading: Documented behavior for how credentials are loaded.
    • Request Integrity: Request canonicalization and signing behavior.
    • Credential Management: Caching, isolation, and redaction of Reqsign-owned credentials and signing material.
    • Runtime Defaults: Documented behavior of provided runtime capabilities and defaults.

    Host Application Responsibilities

    The application embedding Reqsign (e.g., OpenDAL or a custom service) remains responsible for the broader security context:

    • Identity & Access: End-user authentication and authorization.
    • Data Policy: Storage path policy and namespace mapping.
    • Configuration: Choice of endpoint, service, region, account, signer, credential provider, and context adapter.
    • Exposure Control: Deciding whether signed requests or presigned URLs are exposed to end users.
    • Deployment Security: Network egress policies, TLS configuration, process sandboxing, and resource limits.
    • Input Validation: Validating all inputs before they reach Reqsign.

  12. Ensure secure credential handling and redaction

    main

    When implementing custom logic or debugging, be aware of how Reqsign handles sensitive data:

    • Redaction: Reqsign's Debug implementations for credentials redact secret fields.
    • Error Handling: Errors are designed to identify the type of failure (e.g., provider, file path, status code) without including the actual secret-bearing payloads or full responses.
    • Sensitive Data: While Reqsign redacts secrets, it does not treat request paths, endpoint hostnames, regions, account names, or provider names as credentials by default. If your deployment requires higher privacy for these fields, you must configure your logging and tracing sinks accordingly.