DOCSEARCHbyAlgolia
Submit Docs

Powertools for AWS Lambda (Python) Documentation

repository·develop·Indexed 25 days ago

https://github.com/aws-powertools/powertools-lambda-python

A developer toolkit for implementing Serverless best practices in AWS Lambda environments. It provides core utilities for Tracing (AWS X-Ray), structured Logging, and Metrics (CloudWatch EMF), alongside specialized utilities for event handling, idempotency, batch processing, parameter retrieval from SSM/Secrets Manager/DynamoDB, feature flags via AppConfig, and S3 object streaming.

Tokens
69.7K
Snippets
70
Records
487
Agent score
85%

What's inside Powertools for AWS Lambda (Python)

  1. Overview of Data Masking utility

    develop

    The Data Masking utility provides tools to protect data confidentiality by performing three primary operations on sensitive information:

    1. Erasing: Irreversibly replaces sensitive information with a non-sensitive placeholder (default is *****) or a custom mask. This is a one-way, in-memory operation.
    2. Encrypting: Transforms plaintext into ciphertext using an encryption algorithm and a cryptographic key, allowing only authorized personnel to access the original data.
    3. Decrypting: Transforms ciphertext back into its original plaintext using the correct decryption key.

    The utility integrates with the AWS Encryption SDK to follow industry and AWS security best practices, specifically utilizing Envelope Encryption (using a master key to encrypt a data key, which in turn encrypts the sensitive data).

  2. Overview of Amazon CloudWatch EMF Metrics

    develop

    The Metrics utility creates custom metrics asynchronously by logging them to standard output following the Amazon CloudWatch Embedded Metric Format (EMF). This approach allows you to visualize metrics in the Amazon CloudWatch Console without needing custom infrastructure stacks.

    Key capabilities include:

    • Aggregating up to 100 metrics in a single CloudWatch EMF object.
    • Validation against common mistakes (e.g., incorrect metric units, values, or dimension limits).
    • Asynchronous metric creation via the CloudWatch service.
    • Support for context managers to create one-off metrics with specific dimensions.

  3. Overview of Powertools Features

    develop

    Powertools for AWS Lambda (Python) provides a suite of utilities to implement serverless best practices. Key features include:

    • Tracer: Decorators for tracing Lambda handlers and functions.
    • Logger: Structured logging with Lambda context enrichment.
    • Metrics: Asynchronous custom metrics via CloudWatch EMF.
    • Event Handler: Handlers for API Gateway, ALB, Lambda Function URL, VPC Lattice, AppSync, and Bedrock Agents.
    • Parameters: Caching for Parameter Store, Secrets Manager, AppConfig, or DynamoDB.
    • Parser: Data parsing and deep validation using Pydantic.
    • Batch Processing: Handling partial failures for SQS, Kinesis, and DynamoDB Streams.
    • Idempotency: Preventing duplicate execution.
    • Feature Flags: Rule engine for feature toggling.
    • Validation: JSON Schema validator for events and responses.
    • Data Masking: Protecting confidential data via removal or encryption.
    • Streaming: Streaming large datasets.
    • Middleware Factory: Custom middleware creation.
    • Typing: Static typing classes.
    • Event Source Data Classes: Schema descriptions for common triggers.
    • JMESPath Functions: Deserializing encoded JSON payloads.
    • Kafka: Deserializing/validating Kafka events (Avro, Protobuf, JSON Schema).

  4. Overview of Continuous Integration practices

    develop
    Powertools for AWS Lambda (Python) uses an automated pipeline to ensure quality and security throughout the development lifecycle. Checks are categorized into four stages: Pre-commit (local/fast feedback), Pre-Pull Request (local/deeper analysis), Pull Request (CI-level verification), and After merge (comprehensive/long-running checks). This multi-layered approach aims to detect issues early and reduce cognitive overhead for reviewers.

  6. Overview of the Parser utility

    develop

    The Parser utility simplifies data parsing and validation by leveraging Pydantic. It allows you to define data models using pure Python classes to parse and validate incoming Lambda event payloads, ensuring you extract only the required data.

    Key features include:

    • Defining data models using Python classes.
    • Parsing and validating Lambda event payloads.
    • Built-in support for common AWS event sources.
    • Runtime type checking with user-friendly error messages.
    • Compatibility with Pydantic v2.x.

  7. Overview of the Idempotency utility

    develop

    The Idempotency utility for AWS Lambda (Python) ensures that retrying an operation with the same input within a specific time window produces the same output without causing additional side effects.

    Key capabilities:

    • Result Reuse: Returns the previous successful result when a function is called repeatedly with the same idempotency key.
    • Flexible Key Selection: Allows you to derive the idempotency key from the entire payload or specific fields within it.
    • Safety Mechanisms: Protects against concurrent requests, timeouts, missing keys, and payload tampering.
    • Multiple Persistence Layers: Supports Amazon DynamoDB, Valkey, Redis OSS, or any Redis-compatible cache to store idempotency records.

  8. Overview of Powertools for AWS Lambda (Python) features

    develop

    Powertools for AWS Lambda (Python) provides a suite of utilities to increase developer velocity and implement best practices.

    Core Utilities:

    • Tracing: Decorators for tracing Lambda handlers and functions (sync/async).
    • Logging: Structured logging enriched with Lambda context.
    • Metrics: Asynchronous custom metrics via CloudWatch Embedded Metric Format (EMF).
    • Event Handlers: Specialized handlers for AppSync, API Gateway, ALB, Lambda Function URL, VPC Lattice, and Amazon Bedrock Agents.

    Additional Utilities:

    • Middleware factory: Create custom middleware for pre/post invocation logic.
    • Parameters: Retrieve and cache values from Parameter Store, Secrets Manager, AppConfig, or DynamoDB.
    • Batch processing: Handle partial failures for SQS, Kinesis, and DynamoDB Streams.
    • Idempotency: Ensure Lambda functions are safe to retry.
    • Validation & Parsing: JSON Schema validation and Pydantic-based data parsing.
    • Data Masking: Protect confidential data via removal or encryption.
    • Feature Flags: Rule engine for feature toggling based on input.
    • Other utilities: Typing, Event source data classes, Streaming, Kafka deserialization, and JMESPath functions.

  9. Overview of Feature Flags utility

    develop

    The Feature Flags utility provides a rule engine to dynamically enable or disable features based on input. It supports both static flags (simple on/off toggles like TRACER_ENABLED=True) and dynamic flags (varying states based on context, such as enabling premium features for specific customers).

    Key capabilities include:

    • Defining simple feature flags to decide when to enable features.
    • Fetching one or all enabled flags for a given application context.
    • Supporting static feature flags.
    • Supporting time-based feature flags.
    • Allowing custom Feature Flags Store Providers.

    Note: When using AppConfigStore, only AWS AppConfig freeform configuration profiles are currently supported. For purely static flags, you may also consider using the Parameters utility.

  11. Use SSM Parameter Store with Powertools for AWS Lambda (Python)

    develop
    The aws_lambda_powertools.utilities.parameters.ssm module provides a high-level utility to fetch parameters from the AWS Systems Manager (SSM) Parameter Store. It simplifies the process of retrieving configuration values, secrets, or other parameters directly within your Lambda function, handling the underlying Boto3 calls and potentially providing caching mechanisms (depending on the specific implementation details of the utility).