MinIO Go Client SDK

repository·master·Indexed 25 days ago

https://github.com/minio/minio-go

A Go client SDK providing APIs to access Amazon S3 compatible object storage. It includes functionality for bucket and object operations, presigned URLs, server-side encryption, lifecycle management, CORS configuration, bucket notifications, IAM policies, and Quality of Service (QoS) settings.

Tokens
45.2K
Snippets
108
Records
276
Agent score
83%

What's inside minio-go

  1. Initialize a MinIO Client object

    master

    To interact with a MinIO or S3-compatible server, you must initialize a *minio.Client using the minio.New function. This requires an endpoint string and a pointer to a minio.Options struct. You can provide credentials using credentials.NewStaticV4 and specify whether to use SSL via the Secure field in the options.

    package main
    
    import (
    	"log"
    
    	"github.com/minio/minio-go/v7"
    	"github.com/minio/minio-go/v7/pkg/credentials"
    )
    
    func main() {
    	endpoint := "play.min.io"
    	accessKeyID := "Q3AM3UQ867SPQQA43P2F"
    	secretAccessKey := "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
    	useSSL := true
    
    	// Initialize minio client object.
    	minioClient, err := minio.New(endpoint, &minio.Options{
    		Creds:  credentials.NewStaticV4(accessKeyID, secretAccessKey, ""),
    		Secure: useSSL,
    	})
    	if err != nil {
    		log.Fatalln(err)
    	}
    
    	log.Printf("%#v\n", minioClient) // minioClient is now setup
    }
  2. Initialize a MinIO Client for AWS S3

    master

    To connect to an Amazon S3 compatible storage service, use the minio.New constructor. You must provide the service endpoint and a minio.Options struct containing your credentials and security settings.

    package main
    
    import (
    	"fmt"
    
    	"github.com/minio/minio-go/v7"
    	"github.com/minio/minio-go/v7/pkg/credentials"
    )
    
    func main() {
    	// Initialize minio client object.
    	s3Client, err := minio.New("s3.amazonaws.com", &minio.Options{
    		Creds:  credentials.NewStaticV4("YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", ""),
    		Secure: true,
    	})
    	if err != nil {
    		fmt.Println(err)
    		return
    	}
    }
  3. Manage multipart uploads

    master

    For large objects, use the multipart upload workflow:

    1. Initiate: Call NewMultipartUpload to start an upload and receive an uploadID.
    2. Upload Parts: Use PutObjectPart to upload individual chunks of the data.
    3. Complete: Call CompleteMultipartUpload with a slice of CompletePart to concatenate the parts and commit the object.
    4. Abort: If an upload fails or is no longer needed, call AbortMultipartUpload to clean up.

    Additional Multipart Operations:

    • ListMultipartUploads: List incomplete uploads.
    • ListObjectParts: List parts already uploaded for a specific uploadID.
    • CopyObjectPart: Create a part in a multipart upload by copying a portion of an existing object.
  4. Upload large objects using multipart streaming

    master

    The MinIO Go client supports uploading large objects via multipart streaming. Depending on the type of reader provided, the client automatically selects the most efficient upload strategy:

    1. Parallel Multipart Upload: If PutObjectOptions.ConcurrentStreamParts is true and NumThreads > 1, the client performs a parallel multipart upload.
    2. ReadAt Multipart Upload: If the reader implements io.ReaderAt (and is not a *minio.Object), the client uses a strategy that allows reading at specific offsets to avoid re-reading data, which is highly efficient for resuming or parallelizing parts.
    3. Sequential Multipart Upload: For standard io.Reader types, the client performs a sequential multipart upload.
    4. Single PutObject: For Google Cloud Storage or when multipart is unavailable/denied, the client may fall back to a single PutObject operation.

    To ensure optimal performance, you can configure PartSize and NumThreads in PutObjectOptions.

  5. Configure a PostPolicy for browser uploads

    master
    The PostPolicy type provides a way to construct Amazon S3 POST policies in JSON format. These policies are used to restrict and validate browser-based uploads (via HTTP POST). You can set constraints on the bucket, object key, content type, file size, and more. Use NewPostPolicy() to initialize a new policy and then chain various Set* methods to define your constraints.
  6. Copy a partial object with conditions using CopyObject

    master

    You can use CopyObject to copy only a specific part of a source object (using Start and End byte offsets) and apply conditions such as matching an ETag or specific modification time ranges.

    // Use-case 2:
    // Copy object with copy-conditions, and copying only part of the source object.
    // 1. that matches a given ETag
    // 2. and modified after 1st April 2014
    // 3. but unmodified since 23rd April 2014
    // 4. copy only first 1MiB of object.
    
    // Source object
    srcOpts := minio.CopySrcOptions{
    	Bucket:               "my-sourcebucketname",
    	Object:               "my-sourceobjectname",
    	MatchETag:            "31624deb84149d2f8ef9c385918b653a",
    	MatchModifiedSince:   time.Date(2014, time.April, 1, 0, 0, 0, 0, time.UTC),
    	MatchUnmodifiedSince: time.Date(2014, time.April, 23, 0, 0, 0, 0, time.UTC),
    	Start:                0,
    	End:                  1024*1024 - 1,
    }
    
    // Destination object
    dstOpts := minio.CopyDestOptions{
    	Bucket: "my-bucketname",
    	Object: "my-objectname",
    }
    
    // Copy object call
    _, err = minioClient.CopyObject(context.Background(), dstOpts, srcOpts)
    if err != nil {
    	fmt.Println(err)
    	return
    }
    
    fmt.Println("Successfully copied object:", uploadInfo)
  7. Example: Create a Bucket and Upload a File

    master

    This example demonstrates how to connect to a MinIO server, create a new bucket with a specific region, and upload a file from the local filesystem using FPutObject.

    // FileUploader.go MinIO example
    package main
    
    import (
    	"context"
    	"log"
    
    	"github.com/minio/minio-go/v7"
    	"github.com/minio/minio-go/v7/pkg/credentials"
    )
    
    func main() {
    	ctx := context.Background()
    	endpoint := "play.min.io"
    	accessKeyID := "Q3AM3UQ867SPQQA43P2F"
    	secretAccessKey := "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
    	useSSL := true
    
    	// Initialize minio client object.
    	minioClient, err := minio.New(endpoint, &minio.Options{
    		Creds:  credentials.NewStaticV4(accessKeyID, secretAccessKey, ""),
    		Secure: useSSL,
    	})
    	if err != nil {
    		log.Fatalln(err)
    	}
    
    	// Make a new bucket called testbucket.
    	bucketName := "testbucket"
    	location := "us-east-1"
    
    	err = minioClient.MakeBucket(ctx, bucketName, minio.MakeBucketOptions{Region: location})
    	if err != nil {
    		// Check to see if we already own this bucket (which happens if you run this twice)
    		exists, errBucketExists := minioClient.BucketExists(ctx, bucketName)
    		if errBucketExists == nil && exists {
    			log.Printf("We already own %s\n", bucketName)
    		} else {
    			log.Fatalln(err)
    		}
    	} else {
    		log.Printf("Successfully created %s\n", bucketName)
    	}
    
    	// Upload the test file
    	// Change the value of filePath if the file is in another location
    	objectName := "testdata"
    	filePath := "/tmp/testdata"
    	contentType := "application/octet-stream"
    
    	// Upload the test file with FPutObject
    	info, err := minioClient.FPutObject(ctx, bucketName, objectName, filePath, minio.PutObjectOptions{ContentType: contentType})
    	if err != nil {
    		log.Fatalln(err)
    	}
    
    	log.Printf("Successfully uploaded %s of size %d\n", objectName, info.Size)
    }
  8. List bucket inventory configurations using an iterator

    master

    For Go 1.23+, use ListBucketInventoryConfigurationsIterator to iterate through all inventory configurations for a bucket using the iter.Seq2 pattern.

    for config, err := range minioClient.ListBucketInventoryConfigurationsIterator(context.Background(), "mybucket") {
    	if err != nil {
    		log.Fatalln(err)
    		break
    	}
    	fmt.Printf("Inventory ID: %s, Bucket: %s\n", config.ID, config.Bucket)
    }
  9. Configure bucket replication

    master

    Manage bucket replication settings. Note that for MinIO, the Role must be obtained by first defining the replication target using mc admin bucket remote set to associate source and destination buckets.

    • SetBucketReplication(ctx context.Context, bucketName string, cfg replication.Config) error: Sets the replication configuration.
    • GetBucketReplication(ctx context.Context, bucketName string) (replication.Config, error): Retrieves the current replication configuration.
    • RemoveBucketReplication(ctx context.Context, bucketName string) error: Removes the replication configuration from the bucket.
    // Example: Setting replication configuration
    replicationStr := `<ReplicationConfiguration>
       <Role></Role>
       <Rule>
          <DeleteMarkerReplication>
             <Status>Disabled</Status>
          </DeleteMarkerReplication>
          <Destination>
             <Bucket>string</Bucket>
             <StorageClass>string</StorageClass>
          </Destination>
          <Filter>
             <And>
                <Prefix>string</Prefix>
                <Tag>
                   <Key>string</Key>
                   <Value>string</Value>
                </Tag>
                ...
             </And>
             <Prefix>string</Prefix>
             <Tag>
                <Key>string</Key>
                <Value>string</Value>
             </Tag>
          </Filter>
          <ID>string</ID>
          <Prefix>string</Prefix>
          <Priority>integer</Priority>
          <Status>string</Status>
       </Rule>
    </ReplicationConfiguration>`
    replicationConfig := replication.Config{}
    if err := xml.Unmarshal([]byte(replicationStr), &replicationConfig); err != nil {
    	log.Fatalln(err)
    }
    replicationConfig.Role = "arn:minio:s3::598361bf-3cec-49a7-b529-ce870a34d759:*"
    err = minioClient.SetBucketReplication(context.Background(), "my-bucketname", replicationConfig)
  10. Manage bucket versioning

    master

    Control bucket versioning support using the following methods:

    • EnableVersioning(ctx context.Context, bucketName string) error: Enables versioning support for the bucket.
    • SuspendVersioning(ctx context.Context, bucketName string) error: Suspends versioning support for the bucket.
    • GetBucketVersioning(ctx context.Context, bucketName string) (BucketVersioningConfiguration, error): Retrieves the current versioning configuration for the bucket.
    // Enable versioning
    err := minioClient.EnableVersioning(context.Background(), "my-bucketname")
    
    // Suspend versioning
    err := minioClient.SuspendVersioning(context.Background(), "my-bucketname")
    
    // Get versioning config
    versioningConfig, err := s3Client.GetBucketVersioning(context.Background(), "my-bucketname")
  11. List bucket inventory configurations with pagination

    master

    Use ListBucketInventoryConfigurations to retrieve up to 100 inventory configurations for a bucket. If NextContinuationToken is not empty, use it in a subsequent call to fetch the next page of results.

    result, err := minioClient.ListBucketInventoryConfigurations(context.Background(), "mybucket", "")
    if err != nil {
    	log.Fatalln(err)
    }
    for _, item := range result.Items {
    	fmt.Printf("Inventory ID: %s\n", item.ID)
    }
    if result.NextContinuationToken != "" {
    	// Fetch next page
    	nextResult, _ := minioClient.ListBucketInventoryConfigurations(context.Background(), "mybucket", result.NextContinuationToken)
    }