DOCSEARCHbyAlgolia
Submit Docs

s3transfer

repository·develop·Indexed 42 days ago

https://github.com/boto/s3transfer

An Amazon S3 Transfer Manager for Python designed to simplify and optimize the transfer of data to and from Amazon S3. It supports multipart and nonmultipart scenarios for uploads, downloads, and copies, including lifecycle management for failed or cancelled transfers.

Tokens
560
Snippets
0
Records
4
Agent score
4%

What's inside s3transfer

  1. Overview of s3transfer

    develop

    s3transfer is a Python library designed for managing Amazon S3 transfers. It is maintained and published by Amazon Web Services.

    Note for Production Use: This project is not currently General Availability (GA). If you intend to use it in a production environment, it is recommended to lock your dependency to a specific minor version, as interfaces may change between minor releases. For a more stable, GA interface for S3 operations, consider using the interfaces provided by boto3.

  2. Supported S3 Transfer Scenarios: Copies

    develop

    The S3 transfer manager supports copying single or multiple objects, both multipart and nonmultipart. It manages the multipart copy lifecycle to prevent orphaned parts upon failure or cancellation.

    Supported copy features include:

    • Copying specific object versions
    • Using requester pays
    • Using sse-c keys (including copy source sse-c keys)
    • Using copy source parameters: CopySourceIfMatch, CopySourceIfModifiedSince, CopySourceIfNoneMatch, and CopySourceIfUnmodifiedSince
    • Metadata directives (applied to nonmultipart objects)
    • Providing a transfer size in lieu of using HeadObject

  3. Supported S3 Transfer Scenarios: Uploads

    develop

    The S3 transfer manager supports various upload scenarios, including single and multiple files, both multipart and nonmultipart. It handles multipart upload lifecycle management, ensuring that failed or cancelled uploads abort and clean up orphaned parts (e.g., failures in UploadPart, CompleteMultipartUpload, or local read failures).

    Supported extra parameters for uploads include:

    • ACLs
    • CacheControl
    • ContentDisposition
    • ContentEncoding
    • ContentLanguage
    • ContentType
    • Expires
    • Metadata
    • Grants
    • StorageClass
    • SSE (including KMS)
    • sse-c key
    • Requester pays

  4. Supported S3 Transfer Scenarios: Downloads

    develop

    The S3 transfer manager supports downloading single or multiple objects, both multipart and nonmultipart. Downloads are written to a temporary file and renamed to the final filename only after the object is completely downloaded; failed downloads trigger automatic cleanup of the temporary file.

    Supported download features include:

    • Downloading specific object versions
    • Downloading objects encrypted with sse-c
    • Using requester pays
    • Retrying on connection-related errors
    • Providing a transfer size in lieu of using HeadObject