Amazon Bedrock Samples

Amazon Bedrock Model Distillation is a workflow that automates the creation of smaller, faster, and more cost-efficient models. It delivers use-case specific accuracy comparable to larger, more capable 'teacher' models.

Key Features:

• Automated Workflow: Automates teacher response generation, data synthesis, and student model fine-tuning with optimized hyperparameter tuning.
• No Labeling Required: Removes the need to manually create labeled datasets for fine-tuning.
• Efficiency: Provides high accuracy while maintaining the speed of smaller models.
• Cost Optimization: Reduces inference costs compared to using larger, advanced models.

Common Use Cases:

• Retrieval-Augmented Generation (RAG)
• Document Summarization
• Chatbot Deployments
• Text Classification

This pipeline implements a four-stage process to distill knowledge from large language models (LLMs) into smaller, specialized models specifically optimized for citation-aware question answering. The goal is to create efficient models that maintain high-quality citation capabilities.

The pipeline consists of the following stages:

1. Data Preparation: Uses the SQuAD v2.0 dataset and implements a structured XML output format to handle both answerable and 'impossible' questions.
2. Model Distillation: Transfers knowledge from a teacher model to a student model.
3. Batch Inference: Executes large-scale inference using distributed processing and retry mechanisms.
4. Evaluation: Assesses the distilled model using metrics like Citation Coverage, Correctness, Completeness, Faithfulness, Helpfulness, and Logical Coherence.

Prompt engineering is the practice of optimizing the quality and performance of a foundation model's response to a request. When working with Amazon Bedrock, effective prompt engineering typically involves the following techniques:

• Word choice and Phrasing: Selecting specific language to guide the model.
• Few-shot learning: Providing specific examples within the prompt to demonstrate the desired output.
• Structure: Using line breaks and content separators to organize information.
• Format Alignment: Following established formats that align with how the specific model was trained.
• Stop Sequences: Using stop sequences to instruct the model on when to cease text generation.

This workshop demonstrates how to build a multimodal data processing pipeline using the Bedrock Data Automation (BDA) service to analyze raw podcast audio files. The goal is to transform unstructured audio into structured, actionable data for downstream analysis.

Key capabilities explored include:

• Transcription: Extracting textual transcripts from raw audio.
• Speaker Identity: Identifying different speakers within the audio.
• Content Moderation: Analyzing audio for specific content types.
• Sentiment Analysis: Determining the emotional tone of the discussion.
• Summarization: Generating quick summaries of podcast content.
• Metadata Extraction: Incorporating timestamps, speaker information, and chapter segmentation into the pipeline.

This implementation demonstrates a Retrieval Augmented Generation (RAG) approach that leverages multiple data sources to answer queries. It uses a MultiRetrievalQAChain to combine information from:

1. Unstructured Data: Retrieved from an Amazon Bedrock knowledge base (e.g., PDFs, text files).
2. Structured Data: Retrieved from a database (e.g., Amazon Athena) using a Text-to-SQL approach.

The system uses the Claude 3.0 Sonnet language model to generate responses by synthesizing information from these sources while maintaining conversation context via a memory buffer.

Bedrock-ICYM is a Python-based observability and evaluation solution designed for Amazon Bedrock applications. It provides a way to log and track metrics for Knowledge Bases, Agents, InvokeModel calls, Function Calling, and Guardrails.

Data is ingested via Amazon Kinesis Data Firehose into an Amazon S3 bucket, which enables downstream analysis using Amazon Athena and visualization via Amazon QuickSight. The solution uses a decorator-based approach to minimize integration effort and supports both production (Firehose) and local (in-memory) logging modes.

Function calling allows you to provide Large Language Models (LLMs) with descriptions of tools and functions. The LLM can then intelligently decide when and how to invoke these tools based on a user's query to help answer questions or complete complex tasks.

This repository provides several implementation patterns for function calling using different Amazon Bedrock interfaces and third-party frameworks.

The Amazon Titan Multimodal Embeddings G1 model allows you to generate embeddings for both text and image data. These embeddings enable multimodal search and Retrieval Augmented Generation (RAG) workflows, such as:

• Searching for images using text queries.
• Searching for images using image queries.
• Searching using a combination of both text and images.