MinIO Python Client SDK

For high-performance workloads, put_object and get_object can dispatch to MinIO's RDMA + GPUDirect Storage path via minio-cpp. This is an opt-in feature.

Requirements

• libminiocpp.so must be on the host's library search path (or specified via the MINIOCPP_LIB environment variable).
• libminiocpp.so must be built with -DMINIO_CPP_ENABLE_RDMA=ON.

Usage

1. Set enable_rdma=True when initializing the Minio client.
2. Use a buffer-protocol object (like bytearray) for the data= argument in put_object or the into= argument in get_object to trigger RDMA.
3. For GPU buffers (e.g., CuPy or PyTorch), pass the raw pointer as an int to data= or into=.

Warning: The SDK remains pure-Python unless these specific conditions are met.

from minio import Minio

client = Minio(
    endpoint="server:9000",
    access_key="...",
    secret_key="...",
    secure=False,
    enable_rdma=True,             # opt-in
)

buf = bytearray(1 << 20)
client.put_object(
    bucket_name="b", object_name="o",
    data=buf, length=len(buf),    # buffer-protocol object selects RDMA
)

dst = bytearray(1 << 20)
n = client.get_object(
    bucket_name="b", object_name="o",
    into=dst, length=len(dst),    # into= selects RDMA, returns bytes
)

When using put_object or fput_object, you can specify how the data is encrypted on the server using the sse parameter. Supported types include:

• S3-managed keys: Use SseS3().
• KMS-managed keys: Use SseKMS(key_id, context) where key_id is the KMS key ID and context is a dictionary of encryption context.
• Customer-provided keys: Use SseCustomerKey(key) where key is a 32-byte secret key.

# Upload data with customer key type of server-side encryption.
result = client.put_object(
    bucket_name="my-bucket",
    object_name="my-object",
    data=io.BytesIO(b"hello"),
    length=5,
    sse=SseCustomerKey(b"32byteslongsecretkeymustprovided"),
)

# Upload data with KMS type of server-side encryption.
result = client.put_object(
    bucket_name="my-bucket",
    object_name="my-object",
    data=io.BytesIO(b"hello"),
    length=5,
    sse=SseKMS("KMS-KEY-ID", {"Key1": "Value1", "Key2": "Value2"}),
)

# Upload data with S3 type of server-side encryption.
result = client.put_object(
    bucket_name="my-bucket",
    object_name="my-object",
    data=io.BytesIO(b"hello"),
    length=5,
    sse=SseS3(),
)

This example demonstrates how to connect to a MinIO service, create a bucket (handling cases where it already exists), and upload a local file to that bucket using fput_object.

# 引入MinIO包。
from minio import Minio
from minio.error import (ResponseError, BucketAlreadyOwnedByYou,
                         BucketAlreadyExists)

# 使用endpoint、access key和secret key来初始化minioClient对象。
minioClient = Minio('play.min.io',
                    access_key='Q3AM3UQ867SPQQA43P2F',
                    secret_key='zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG',
                    secure=True)

# 调用make_bucket来创建一个存储桶。
try:
       minioClient.make_bucket("maylogs", location="us-east-1")
except BucketAlreadyOwnedByYou as err:
       pass
except BucketAlreadyExists as err:
       pass
except ResponseError as err:
       raise
else:
        try:
               minioClient.fput_object('maylogs', 'pumaserver_debug.log', '/tmp/pumaserver_debug.log')
        except ResponseError as err:
               print(err)

This example demonstrates how to connect to a MinIO server, check if a bucket exists (creating it if necessary), and upload a local file to a specific bucket using fput_object.

Steps:

1. Initialize the Minio client.
2. Use client.bucket_exists(bucket_name=...) to check for the bucket.
3. Use client.make_bucket(bucket_name=...) to create the bucket if it is missing.
4. Use client.fput_object(...) to upload the file from a local path.

Note: This example uses the public MinIO play server. Data uploaded here is public and world-readable.

# file_uploader.py MinIO Python SDK example
from minio import Minio
from minio.error import S3Error

def main():
    # Create a client with the MinIO server playground, its access key
    # and secret key.
    client = Minio(
        endpoint="play.min.io",
        access_key="Q3AM3UQ867SPQQA43P2F",
        secret_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG",
    )

    # The file to upload, change this path if needed
    source_file = "/tmp/test-file.txt"

    # The destination bucket and filename on the MinIO server
    bucket_name = "python-test-bucket"
    destination_file = "my-test-file.txt"
    
    # Make the bucket if it doesn't exist.
    found = client.bucket_exists(bucket_name=bucket_name)
    if not found:
        client.make_bucket(bucket_name=bucket_name)
        print("Created bucket", bucket_name)
    else:
        print("Bucket", bucket_name, "already exists")

    # Upload the file, renaming it in the process
    client.fput_object(
        bucket_name=bucket_name,
        object_name=destination_file,
        file_path=source_file,
    )
    print(
        source_file, "successfully uploaded as object",
        destination_file, "to bucket", bucket_name,
    )

if __name__ == "__main__":
    try:
        main()
    except S3Error as exc:
        print("error occurred.", exc)

Use put_object to upload data from a file-like object (an object with a callable read() method returning bytes) to a bucket. This is useful for streaming data or uploading in-memory buffers.

Key Parameters:

• bucket_name: Name of the destination bucket.
• object_name: Name of the object to create.
• data: An io.BinaryIO object containing the data.
• length: Data size in bytes. Use -1 for unknown size (requires setting part_size).
• content_type: The MIME type of the object (defaults to application/octet-stream).
• metadata: A dictionary for user metadata.
• sse: Server-side encryption configuration (e.g., SseS3, SseKMS, or SseCustomerKey).
• tags: An instance of minio.models.Tags.
• retention: Retention configuration.
• legal_hold: Boolean flag to set legal hold.
• progress: A progress object to track upload status.
• part_size: Multipart part size (required if length is -1).

Returns: minio.models.ObjectWriteResponse containing object_name, etag, and version_id.

import io

# Upload data.
result = client.put_object(
    bucket_name="my-bucket",
    object_name="my-object",
    data=io.BytesIO(b"hello"),
    length=5,
)
print(
    f"created {result.object_name} object; etag: {result.etag}, "
    f"version-id: {result.version_id}",
)

# Upload unknown sized data using a stream.
with urlopen(
    "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.4.81.tar.xz",
) as data:
    result = client.put_object(
        bucket_name="my-bucket",
        object_name="my-object",
        data=data,
        length=-1,
        part_size=10*1024*1024,
    )