Powertools for AWS Lambda (.NET)

repository·develop·Indexed 41 days ago

https://github.com/aws-powertools/powertools-lambda-dotnet

A developer toolkit for .NET developers on AWS designed to implement serverless best practices and increase developer velocity. Version 1.0.0 includes support for Native AOT compilation to improve cold start performance, batch processing for SQS, Kinesis, and DynamoDB streams, and idempotency patterns.

Tokens
75.7K
Snippets
225
Records
328
Agent score
51%

What's inside AWS Lambda Powertools for .NET

  1. Overview of Powertools for AWS Lambda (.NET) features

    develop

    Powertools for AWS Lambda (.NET) is a developer toolkit designed to implement serverless best practices. The core utilities include:

    • Logging: Structured JSON logging that handles serialization, Lambda event payloads, and cold start information.
    • Metrics: Asynchronous custom metric collection using Amazon CloudWatch Embedded Metric Format (EMF).
    • Tracing: Integration with AWS X-Ray for visibility into function calls and external requests, including automatic ColdStart annotations.
    • Parameters: High-level retrieval of values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB.
    • Idempotency: A utility to make Lambda functions idempotent and safe to retry.
    • Batch Processing: Handles partial failures when processing batches from Amazon SQS, Amazon Kinesis Data Streams, and Amazon DynamoDB Streams.
    • Event Handlers: Specialized utilities for parsing and handling AppSync Events and Amazon Bedrock Agent Function events.
  2. Retrieve parameters using AWS.Lambda.Powertools.Parameters

    develop

    The AWS.Lambda.Powertools.Parameters utility allows you to retrieve one or multiple parameter values from AWS Systems Manager (SSM) Parameter Store, AWS Secrets Manager, or Amazon DynamoDB.

    Key capabilities include:

    • Retrieving single or multiple parameters from a provider.
    • Caching parameter values for a configurable duration (defaults to 5 seconds).
    • Transforming parameter values from JSON or base64 encoded strings.
    • Supporting custom parameter store providers.
  3. Use AWS.Lambda.Powertools.Logging for structured JSON logging

    develop

    The AWS.Lambda.Powertools.Logging utility provides an AWS Lambda-optimized logger that outputs logs in structured JSON format. It automatically captures key metadata from the Lambda context, such as cold start status, function name, ARN, and X-Ray trace IDs.

    Key capabilities include:

    • Automatic Context Capture: Includes Lambda context, cold start status, and X-Ray trace IDs in every log.
    • Event Logging: Can be configured to log the incoming Lambda event (disabled by default).
    • Log Sampling: Enables DEBUG level logging for a specific percentage of requests (disabled by default).
    • Dynamic Context: Allows appending additional keys to the structured log at any point during the execution lifecycle.
    // Example of using the Logger in a Lambda function
    public class Function
    {
        [Logging(LogEvent = true)]
        public async Task<APIGatewayProxyResponse> FunctionHandler(APIGatewayProxyRequest apigwProxyEvent, ILambdaContext context)
        {
            // Append keys to add custom context to all subsequent logs in this execution
            var lookupInfo = new Dictionary<string, object>()
            {
                {"LookupInfo", new Dictionary<string, object>{{ "LookupId", apigwProxyEvent.RequestContext.RequestId }}}
            };
            Logger.AppendKeys(lookupInfo);
    
            Logger.LogInformation("Getting ip address from external service");
            // ...
        }
    }
  4. Key features of the Idempotency utility

    develop

    The Idempotency utility provides several capabilities for managing retries and duplicate prevention:

    • Duplicate Prevention: Prevents the Lambda handler from executing more than once on the same event payload during a specific time window.
    • Consistent Results: Ensures the Lambda handler returns the same result when called with the same payload.
    • Granular Key Selection: Allows you to select a subset of the event to use as the idempotency key using JMESPath expressions.
    • Configurable Time Windows: Set a specific duration in which records with the same payload are considered duplicates.
    • Timeout Protection: Automatically expires in-progress executions if the Lambda function times out mid-execution.
    • AOT Support: Supports Ahead-of-Time compilation to native code (Native AOT) starting from version 1.3.0.
  5. Use AWS.Lambda.Powertools.BatchProcessing to handle partial failures

    develop

    The BatchProcessing utility simplifies handling partial failures when processing batches from Amazon SQS, Amazon Kinesis Data Streams, and Amazon DynamoDB Streams.

    Instead of failing an entire batch (which causes all messages to be retried), this utility allows you to report specific failed items. This reduces unnecessary retries and improves efficiency:

    • SQS Standard Queues: Only messages reported as failures return to the queue; successful ones are deleted.
    • Kinesis and DynamoDB Streams: If one failure is reported, it uses that sequence number as the checkpoint. If multiple failures are reported, the lowest sequence number is used as the checkpoint.
  6. Use AWS Lambda Powertools Tracing for AWS X-Ray

    develop

    AWS.Lambda.Powertools.Tracing is a thin wrapper around the AWS X-Ray .NET SDK designed to reduce the overhead of common tracing tasks in AWS Lambda.

    Key capabilities include:

    • Custom Subsegments: Helper methods to simplify creating AWS X-Ray subsegments.
    • Cold Start Tracking: Automatically captures cold starts as annotations.
    • Response/Error Capture: Can capture function responses and full exceptions as metadata.
    • Multi-threading Support: Improved experience when developing with multiple threads.
    • Auto-patching: Automatically patches supported modules by AWS X-Ray.
  7. What is the Idempotency utility and how does it work?

    develop

    The Idempotency utility allows you to convert Lambda functions into idempotent operations that are safe to retry. It ensures that a Lambda handler does not execute more than once for the same event payload within a defined time window and always returns the same result when called with the same payload.

    Core Concepts

    • Idempotency Key: A hash representation of either the entire event or a specific subset of the event (configured via JMESPath). The invocation results are JSON serialized and stored in a persistence layer.
    • Idempotency Record: The data stored in your persistence layer used to coordinate requests. It tracks whether a request is valid, expired, or currently in progress.

    Idempotency Record Schema

    An idempotency record consists of the following fields:

    • IdempotencyKey: The unique hash for the request.
    • Status: An enum representing the state (INPROGRESS, COMPLETED, or EXPIRED).
    • ExpiryTimestamp: When the record is no longer valid.
    • InProgressExpiryTimestamp: Used to expire in-progress executions if the Lambda function times out.
    • ResponseData: The JSON serialized result of the successful invocation.
    • PayloadHash: The hash of the input payload.
  8. What is the Batch Processing utility and how does it work?

    develop

    The Batch Processing utility is designed to handle partial failures when processing batches from Amazon SQS, Amazon Kinesis Data Streams, and Amazon DynamoDB Streams.

    By default, if a Lambda function fails to process any message in a batch, the entire batch is retried. This utility allows you to report specific item failures back to the event source. This behavior changes based on the source:

    • SQS queues: Only messages reported as failures return to the queue for retry; successful ones are deleted.
    • Kinesis and DynamoDB streams: A single reported failure uses its sequence number as the checkpoint. Multiple reported failures use the lowest sequence number as the checkpoint.

    Note: This utility reduces the chance of duplicate processing but does not guarantee it. You should always implement your processing logic in an idempotent manner.

  9. Add one-time extra keys to a single log entry

    develop

    If you want to add metadata to a specific log statement without affecting subsequent logs, use the 'extra keys' functionality. All standard logging methods (e.g., Logger.Information, Logger.Warning) accept a dictionary or keyword arguments that are added to the root structure of that specific log entry only. These keys are not persisted for subsequent messages.

    // Example of adding extra keys to a specific log call
    // Note: The documentation implies passing a dictionary or using keyword arguments
    // which are then merged into the log root.
    Logger.LogInformation("Message with extra context", new Dictionary<string, object> { { "extra_key", "value" } });
  10. Configure Error Handling Policies for Batch Processing

    develop

    You can control how the batch processor reacts to individual item failures using the ErrorHandlingPolicy. This can be set via the POWERTOOLS_BATCH_ERROR_HANDLING_POLICY environment variable or by decorating the handler with the ErrorHandlingPolicy Enum property in the BatchProcessor attribute.

    PolicyDescription
    DeriveFromEventAuto-derive the policy based on the event (Default).
    ContinueOnBatchItemFailureContinue processing the rest of the batch regardless of whether other items fail.
    StopOnFirstBatchItemFailureStop processing other batch items after the first failure. Useful for preserving order.

    Note: When using StopOnFirstBatchItemFailure with parallel processing enabled, all items already scheduled will be allowed to complete before stopping. For strict ordering, use sequential (non-parallel) processing.

  11. Enable and Configure Log Buffering

    develop

    Log buffering allows you to capture logs (at specific levels) in memory and only emit them when certain conditions are met (e.g., an error occurs). This is useful for reducing log noise while retaining detailed debug info for troubleshooting.

    Configuration options for LogBuffering:

    ParameterDescriptionDefault
    EnabledEnable or disable log bufferingFalse
    MaxBytesMaximum size of the log buffer in bytes20480
    BufferAtLogLevelMinimum log level to buffer (Trace, Debug, Information, Warning)Debug
    FlushOnErrorLogAutomatically flush buffer when logging an errorTrue

    Key Behaviors:

    • Scope: Buffers are per-invocation; they do not persist across Lambda invocations.
    • Cold Starts: Logs are never buffered during cold starts.
    • Overflow: If MaxBytes is exceeded, older logs are removed to make room for new ones.
    • Large Logs: If a single log line exceeds MaxBytes, it is emitted directly to stdout and not buffered.
    • Timeouts: If a Lambda times out before flushing, buffered logs are lost.
  12. How AppSync Events Resolver works

    develop

    The AppSync Events resolver acts as a router for real-time WebSocket events. When you call app.resolve(event, context) (or the async version), Powertools automatically parses the incoming AppSync event and routes it to the appropriate handler based on the registered namespace and channel patterns.

    Key routing methods include:

    • OnPublish() / OnPublishAsync(): Handles publish events.
    • OnPublishAggregate() / OnPublishAggregateAsync(): Handles batch/aggregated publish events.
    • OnSubscribe() / OnSubscribeAsync(): Handles subscription events.

    Powertools also handles the response contract, automatically formatting your handler's return value into the specific JSON structure required by AppSync Events.