botocore Documentation

Botocore is a low-level interface to Amazon Web Services (AWS). It serves as the foundational layer for the AWS-CLI and is a key component used by the boto3 project.

Compatibility: Botocore is compatible with Python 3.10 and higher.

Some AWS operations return incomplete result sets that require subsequent requests to retrieve the full data (e.g., S3 list_objects returns up to 1000 objects at a time).

Paginators provide an abstraction over this process. Instead of manually managing markers or tokens to fetch subsequent pages, you use a Paginator object to iterate over the entire result set automatically.

The use of AI tools for assisted development is encouraged, but submissions must follow these rules:

• Human Review: All AI-sourced issues and pull requests must be reviewed by a human before submission.
• Disclosure: Submissions must include a statement such as: generated by AI tools, and reviewed by <person>.
• Quality: Submissions must be genuine improvements. Nuisance PRs or attempts to artificially inflate submission counts are not acceptable and may result in restricted interaction with the repository.

In the old interface, connection pooling was tied to an endpoint object. In the new client interface, connection pooling is tied to the client instance.

To reuse existing HTTP connections and improve performance, use a single client instance for multiple API calls instead of creating new clients for every request.

# Using the same client will reuse any existing HTTP
# connections the client was using.
s3 = session.get_client('s3', 'us-west-2')
for obj in s3.list_objects(Bucket='amzn-s3-demo-bucket')['Contents']:
    name = obj['Key']
    print(s3.head_object(Bucket='amzn-s3-demo-bucket', Key=name))

Unlike the boto library, Botocore client methods require parameters to be passed as **kwargs using CamelCase names. This design choice ensures that the casing of input arguments matches the casing of the response data returned by AWS services, facilitating easier 'round-tripping' (using values from a response directly as inputs for subsequent calls) without manual case conversion. Additionally, the parameter names in Botocore match the names used in official AWS API documentation.

# Botocore usage (CamelCase)
ddb = session.create_client('dynamodb')
ddb.describe_table(TableName='mytable')

# Boto usage (snake_case)
layer1.describe_table(table_name='mytable')

You can install botocore directly via pip, or install it from source using a virtual environment.

Install via pip

$ pip install botocore

Install from source

Assuming you have python and virtualenv installed:

$ git clone https://github.com/boto/botocore.git
$ cd botocore
$ python -m venv .venv
$ source .venv/bin/activate
$ python -m pip install -r requirements.txt
$ python -m pip install -e .

To use pagination, follow these steps:

1. Create a session and a client using botocore.session.get_session().
2. Obtain a reusable Paginator object by calling client.get_paginator('operation_name').
3. Call the .paginate() method on the paginator, passing in the required operation parameters. This returns a PageIterator.
4. Iterate over the PageIterator to process each page of results.

import botocore.session

# Create a session and a client
session = botocore.session.get_session()
client = session.create_client('s3', region_name='us-west-2')

# Create a reusable Paginator
paginator = client.get_paginator('list_objects')

# Create a PageIterator from the Paginator
page_iterator = paginator.paginate(Bucket='amzn-s3-demo-bucket')

for page in page_iterator:
    print(page['Contents'])

Botocore uses Sphinx to generate its documentation. To build the HTML documentation on your local machine, install the necessary requirements and run the Sphinx make command from the docs directory.

$ pip install -r requirements-docs.txt
$ cd docs
$ make html

Before using botocore, you must configure your AWS credentials and default region.

Set up credentials

Create or edit your credentials file (e.g., ~/.aws/credentials) and add your keys:

[default]
aws_access_key_id = YOUR_KEY
aws_secret_access_key = YOUR_SECRET

Set up default region

Create or edit your config file (e.g., ~/.aws/config) to specify a region:

[default]
region=us-east-1

The naming convention for calling operations changes between the two interfaces:

• Old Interface: Operations are retrieved using the service's casing (typically CamelCase), e.g., service.get_operation('ListObjects').
• New Client Interface: Methods are snake_case to follow Python conventions and map 1-to-1 with operation names, e.g., client.list_objects().

# Old
service = session.get_service('s3')
list_objects = service.get_operation('ListObjects')

# New
s3 = session.get_client('s3', 'us-west-2')
list_objects = s3.list_objects