DOCSEARCHbyAlgolia
Submit Docs

AWS GenAI LLM Chatbot

repository·main·Indexed 23 days ago

https://github.com/aws-samples/aws-genai-llm-chatbot

An enterprise-ready generative AI chatbot solution featuring Retrieval Augmented Generation (RAG) capabilities, conversation memory, and a modern React interface. It supports multiple LLM providers including Amazon Bedrock, SageMaker, and GenAIEH Gateway. The solution is deployable via AWS CDK or as a SeedFarmer blueprint, offering flexible configuration for security, data encryption, and model integration via LangChain adapters.

Tokens
66.6K
Snippets
90
Records
371
Agent score
79%

What's inside aws-genai-llm-chatbot

  1. Overview of AWS GenAI LLM Chatbot features

    main

    The AWS GenAI LLM Chatbot is an enterprise-ready solution providing a secure, feature-rich chatbot with Retrieval Augmented Generation (RAG) capabilities.

    Key capabilities include:

    • LLM Support: Integration with Amazon Bedrock (Claude, Llama 2), SageMaker, and custom model endpoints.
    • Gateway Integration: Support for GenAIEH Gateway for extended model access.
    • RAG Implementation: Connection to various data sources for context-aware responses.
    • Security: Fine-grained access controls, audit logging, and data encryption.
    • Memory: Persistent storage for full conversation history.
    • Interfaces: A modern React-based Web UI and API endpoints.
    • Management: Token usage tracking and cost management features.

  2. Overview of GenAI Chatbot on AWS

    main
    The GenAI Chatbot on AWS is a solution designed to help developers experiment with various Large Language Models (LLMs) and Multimodal Language Models (MLLMs). It provides ready-to-use code that allows you to test different model settings, prompts, and configurations directly within your own AWS account. This enables rapid prototyping of generative AI applications, including Retrieval-Augmented Generation (RAG) use cases.

  3. Supported Model Providers

    main

    The solution allows experimentation with a variety of Large Language Models (LLMs) and Multimodal Language Models (MLMs) through the following providers:

    • Amazon Bedrock
    • Amazon SageMaker: Supports self-hosted models from Foundation, Jumpstart, and HuggingFace.
    • Third-party providers via API: Includes Anthropic, Cohere, AI21 Labs, OpenAI, and others via LangChain integrations.

  4. User Interface Architecture

    main

    The solution includes a full-fledged React-based UI with the following characteristics:

    • Hosting: Hosted on Amazon S3 and distributed via Amazon CloudFront.
    • Authentication: Protected by Amazon Cognito.
    • Communication: Uses AWS AppSync (GraphQL) for request management and real-time interaction (messages/responses) via GraphQL subscriptions.
    • Design: Built using the AWS Cloudscape Design System.

  6. Review AWS resources deployed by the solution

    main

    The AWS GenAI LLM Chatbot deploys a variety of resources depending on your pre-deployment configuration. Understanding these resources is critical for managing permissions, security, and costs. The deployment is modular: not all resources listed in the documentation will be deployed unless specific options (like specific RAG engines or private networking) are enabled during configuration.

    Key functional areas include:

    • Authentication: Managed via Amazon Cognito.
    • RAG Engines: Supports Amazon Aurora PostgreSQL (with pgvector), Amazon OpenSearch Serverless, and Amazon Kendra (all optional).
    • Data Ingestion: Uses S3, DynamoDB, Step Functions, Amazon Batch, and Lambda to process files, websites, and RSS feeds.
    • Chatbot API: Powered by AppSync (GraphQL), Lambda, SQS, and SNS.
    • User Interface: A React-based UI hosted on S3 and served via CloudFront (public) or an Application Load Balancer (private VPC deployment).
    • ML Models: Integrates with Amazon Bedrock and Amazon SageMaker.

  7. Enable advanced monitoring features

    main

    Enabling the advanced monitoring setting activates several high-fidelity observability features. Note that this increases costs due to additional CloudWatch metrics, alarms, and AWS X-Ray traces.

    Advanced monitoring includes:

    • AWS X-Ray: Collects traces viewable via the Trace Map in the CloudWatch console.
    • Custom LLM Metrics: For Amazon Bedrock, generates custom metrics per model to track token usage using CloudWatch log filters.
    • CloudWatch Alarms: Creates sample alarms for resource monitoring.

  9. Understand Cognito Federation limitations

    main

    When using Cognito Federation in this solution, be aware of the following constraints:

    • Single Provider: Only one federation provider (plus local Cognito login) can be supported at a time. Multiple federation providers are not supported.
    • OIDC Workflow: Only the OIDC issuer URL is supported; you cannot provide discrete URLs for individual components of the OIDC workflow.
    • SAML Metadata: SAML metadata must be retrieved via a URL. Supplying SAML metadata via local files is not supported.
    • Attribute Mapping: Attribute mapping between the federated provider and Cognito is not handled by the installer and must be configured manually in the Amazon Cognito console after deployment.

  10. Manage user roles and permissions

    main

    The GenAI Chatbot uses roles to define user permissions. Roles are implemented using Amazon Cognito user groups.

    Important: After the first deployment with access control enabled, you must manually assign the admin role to a user to ensure they can access the project. Users without any assigned role will be unable to use the Chatbot.

    Pre-defined Roles

    During deployment, the CDK creates three default roles:

    1. admin: Full access to all features and administrative pages.
    2. workspace_manager: Full access to chatbot and RAG features, but cannot access administrative pages like managing applications.
    3. user: Restricted access; can only access the end-user view of applications they are specifically assigned to.

    Custom Roles and Federation

    • Custom Roles: Instead of using the default user role, you can create new roles in Cognito and associate them with specific applications.
    • Federation: If using Cognito federation with a third-party identity provider, the user's role is assigned during login by reading the custom:chatbot_role user attribute.

  12. Understand the SeedFarmer blueprint directory structure

    main

    The blueprint is organized into several key directories and files that define how the solution is configured and deployed:

    • capability.yaml: The primary definition for the GenAIEH catalog. It defines metadata, input parameters, validation rules, and deployment environments.
    • deployment.yaml: The main SeedFarmer deployment manifest. It defines the deployment structure, target accounts/regions, module groups, and environment variable mapping.
    • seedfarmer.yaml: Contains project metadata like name and description.
    • modules/chatbot/: Contains the core logic for the chatbot module:
      • module.yaml: Defines parameters, output definitions, and stack naming.
      • deployspec.yaml: Contains the actual deployment instructions (commands, environment variable handling, and CDK configuration).
    • assets/: Contains example-input.yaml for configuration reference and visual assets (chatbot-icon.svg, chatbot-thumbnail.svg) for the catalog UI.