Boto3 Documentation

repository·develop·Indexed 27 days ago

https://github.com/boto/boto3

The official Amazon Web Services (AWS) Software Development Kit (SDK) for Python. Boto3 allows developers to integrate Python applications with AWS services such as Amazon S3 and Amazon EC2 through low-level clients and high-level resource collections.

Tokens
58.4K
Snippets
215
Records
267
Agent score
95%

What's inside Boto3

  1. Overview of Boto3 - The AWS SDK for Python

    develop

    Boto3 is the AWS SDK for Python used to create, configure, and manage AWS services like Amazon EC2 and Amazon S3. It provides two primary ways to interact with AWS:

    1. Object-oriented API: A high-level interface for managing services.
    2. Low-level access: Direct access to AWS service APIs.

    Developers commonly refer to the SDK as "Boto3".

  2. Overview of Boto3 specific events

    develop

    Boto3 emits a set of events that users can register to customize clients or resources and modify the behavior of method calls.

    Event Naming Conventions: When registering handlers, you can use keywords in the event name to target specific scopes:

    • service-name: The value used to instantiate a client (e.g., s3).
    • operation-name: The underlying API operation name (e.g., ListObjectsV2). You can retrieve this via client.meta.method_to_api_mapping.
    • resource-name: The name of the resource class (e.g., ServiceResource).

    Conditional Events:

    • creating-resource-class (Order 2): Emitted ONLY when using a service resource.
    • after-call (Order 8): Emitted once the API response is received.
    • after-call-error (Order 9): Emitted when an unsuccessful API response is received.
  3. Understand the Boto3 core features

    develop

    Boto3 provides several distinct interfaces and tools for interacting with AWS services:

    • Resources: A high-level, object-oriented interface.
    • Collections: Tools to iterate and manipulate groups of resources.
    • Clients: Low-level service connections.
    • Paginators: Automatic paging of responses to handle large result sets.
    • Waiters: Mechanisms to block execution until a specific service state is reached.

    Additionally, Boto3 manages sessions, credentials, configuration, authentication, parameter/response handling, an event system for customization, and retry logic.

  4. Understand Boto3 configuration lookup order

    develop

    Boto3 searches for configuration values in a specific order. It uses the first value it finds and stops searching. The lookup order is:

    1. A Config object passed as the config parameter when creating a client.
    2. Environment variables.
    3. The ~/.aws/config file.

    Note that configurations are not wholly atomic; a specific environment variable or a Config object can overwrite individual values found in the AWS config file.

  5. Choose an available retry mode

    develop

    Boto3 provides three retry modes to handle client-side failures and service-side throttling:

    1. legacy (default): Uses an older retry handler.

      • Default max attempts: 5 (includes initial request).
      • Retries on specific connection errors (e.g., ConnectionError, ReadTimeoutError) and throttling exceptions.
      • Uses exponential backoff with a base factor of 2.
    2. standard: A standardized retry logic consistent with other AWS SDKs.

      • Default max attempts: 3 (includes initial request).
      • Includes circuit-breaking functionality to prevent retries during service outages.
      • Retries on an expanded list of transient and throttling errors (e.g., RequestTimeout, TooManyRequestsException, SlowDown).
      • Uses exponential backoff with a base factor of 2 and a maximum backoff time of 20 seconds.
    3. adaptive (experimental): Includes all features of standard mode plus client-side rate limiting.

      • Uses a token bucket and dynamically updates rate-limit variables based on service responses to adapt the client's call rate.
  6. Understand the relationship between Boto3 and Botocore

    develop

    Boto3 is built on top of Botocore, which is also used by the AWS CLI.

    • Botocore provides the low-level clients, session management, and credential/configuration data.
    • Boto3 extends Botocore by adding its own session management, high-level Resources, and Collections.
  7. Configure client-specific settings using the Config object

    develop

    To configure settings that affect only a specific client instance, create a botocore.config.Config object and pass it to the boto3.client() call.

    Key parameters include:

    • region_name (string): The AWS Region for the client. Overrides environment variables and config files, but does not overwrite values explicitly passed to individual service methods.
    • signature_version (string): The signature version (e.g., 'v4'). Use 'v2' for presigned URLs with expiry > 7 days.
    • s3 (dictionary): Amazon S3 service-specific configurations.
    • proxies (dictionary): Maps protocol names (e.g., 'http', 'https') to proxy server addresses.
    • proxies_config (dictionary): Additional proxy settings like proxy_client_cert.
    • retries (dictionary): Configuration for retry mode and max_attempts.
    • client_context_params (dictionary): Service-specific client context parameters.
    import boto3
    from botocore.config import Config
    
    my_config = Config(
        region_name = 'us-west-2',
        signature_version = 'v4',
        retries = {
            'max_attempts': 10,
            'mode': 'standard'
        }
    )
    
    client = boto3.client('kinesis', config=my_config)
  8. Handle multithreading or multiprocessing with Boto3 sessions

    develop

    Session objects are not thread safe and should not be shared across threads or processes. To ensure thread safety, create a new Session object inside each thread or process.

    import boto3
    import boto3.session
    import threading
    
    class MyTask(threading.Thread):
        def run(self):
            # Here we create a new session per thread
            session = boto3.session.Session()
    
            # Next, we create a resource client using our thread's session object
            s3 = session.resource('s3')
    
            # Put your thread-safe code here
  9. Configure nested settings for api_versions and s3

    develop

    Certain configuration settings like api_versions and s3 are nested and require specific indentation when manually editing the ~/.aws/config file.

    Example for api_versions:

    [default]
    region = us-east-1
    api_versions =
        ec2 = 2015-03-01
        cloudfront = 2015-09-17

    Example for s3 settings:

    [default]
    region = us-east-1
    s3 =
        addressing_style = path
        signature_version = s3v4
    [default]
    region = us-east-1
    api_versions =
        ec2 = 2015-03-01
        cloudfront = 2015-09-17
    
    [default]
    region = us-east-1
    s3 =
        addressing_style = path
        signature_version = s3v4
  10. Use AWS IAM Identity Center (SSO) profiles

    develop

    To use IAM Identity Center (formerly AWS SSO), first configure your SSO profiles using the AWS CLI (v2). Once configured, you can use the profile in Boto3 by specifying the profile_name in a Session or setting the AWS_PROFILE environment variable.

    import boto3
    
    # Using a profile configured for IAM Identity Center
    session = boto3.Session(profile_name='my-sso-profile')
    s3_client = session.client('s3')
  11. Generate a presigned URL for S3 object access

    develop

    Use generate_presigned_url to grant temporary access to an S3 object to users without AWS credentials. It is recommended to configure the S3 client with Signature Version 4 (s3v4), the specific AWS region, and virtual-hosted style addressing (s3={'addressing_style': 'virtual'}).

    To download the object, the recipient can use the returned URL in a browser or via an HTTP GET request.