GenAIExamples Documentation

Visual Question Answering (VQA) is a task where a model answers open-ended questions based on an input image. The input is a combination of an image and a text question, and the output is a natural language answer.

Common use cases include:

• Accessibility: Assisting visually impaired individuals.
• Education: Interactive learning with visual materials.
• Customer Service/E-commerce: Answering user questions about products.
• Image Retrieval: Finding images based on specific visual characteristics (e.g., "Is there a dog?").

The Arbitration Post-Hearing Assistant is a GenAI module that processes post-hearing transcripts or arbitration documents to extract structured insights.

Typical Workflow:

1. Upload/Stream: Provide a post-hearing transcript.
2. LLM Analysis: An LLM backend (integrating with vLLM or TGI) analyzes the text.
3. Structured Output: The system returns a JSON object containing extracted entities and a summary.

Extracted Entities include:

• Case number
• Parties involved (claimant/respondent)
• Arbitrator(s)
• Hearing date and time
• Next hearing schedule and purpose
• Hearing outcomes and reasons

AudioQnA is a voice-enabled chat interface with the following capabilities:

• Talking Chat: Users can initiate voice chats, provide voice input, and customize the conversation sound using uploaded files.
• Auto-scroll: The chat interface automatically scrolls to the bottom as new messages arrive.

The ChatQnA Customized UI provides a full-featured interface for interacting with generative AI models. Key capabilities include:

• Text Chat: Initiate conversations via text input. Dialogue content can be customized by uploading files.
• File Management: Upload files locally or via remote links to use as a knowledge base. You can also delete specific uploaded files.
• Chat History: Historical records are retained even after page refreshes.
• Session Controls: Clear the current dialog box to start fresh. The UI tracks 'End to End Time' to show the duration of the current conversation.
• Navigation: The chat automatically scrolls to the bottom on new messages, and users can manually scroll to the top using a dedicated icon.

The Code Translation Application (CodeTrans) is an end-to-end workflow designed to convert code from one programming language to another (e.g., Python to Java, or JavaScript to TypeScript) while preserving original functionality. It leverages Large Language Models (LLMs) to perform source-to-source translation or transpilation.

The application is built using component-level microservices defined in the GenAIComps repository and demonstrates Text Generation Inference across various hardware platforms.

The Code Generation (CodeGen) example is an AI application designed to assist developers by generating computer code from natural language prompts or existing code context. It uses Large Language Models (LLMs) to provide capabilities such as code completion, translation, summarization, refactoring, and error detection.

Architecture

The application uses a microservice-based architecture. A gateway orchestrates requests between the user interface and backend services, which include:

• Core LLM: For code generation.
• Retrieval-Augmented Generation (RAG): For providing context-aware responses using a vector database.
• CodeGen-MegaService: A collection of microservices including Embedding, Retrieval, Agents, and the LLM service.

The Finance Agent is a hierarchical multi-agent system designed for financial document processing and analysis. It automates three primary workflows:

1. Summarization: Summarizing long financial documents and extracting key points using the OPEA DocSum microservice.
2. Question Answering: Answering specific queries about financial documents (e.g., SEC filings) using a dedicated worker agent.
3. Investment Research: Conducting research on public companies to generate comprehensive investment reports using a worker agent.

Users interact with the system via a graphical user interface (UI) to submit requests or upload documents for processing. A supervisor agent manages the workflow by delegating tasks to either worker agents or the summarization microservice.

The Document Summarization Application is a use case designed to create summaries of various document types, including news articles, research papers, technical documents, legal documents, and multimedia files (PDFs, Notion pages, etc.).

It leverages LangChain to implement summarization strategies and uses Text Generation Inference (TGI) to facilitate LLM inference. The application is built using component-level microservices defined in GenAIComps.

VideoQnA is a framework designed to retrieve videos based on user prompts. It utilizes video embeddings to perform vector similarity searches within Intel's VDMS vector database. The system is optimized to run entirely on Intel Xeon CPUs and supports long-form videos and time-based search capabilities.

Key technical characteristics:

• RAG Workflow: Implements Retrieval-Augmented Generation using LangChain, Intel VDMS VectorDB, and Text Generation Inference.
• Video Processing: Converts videos into feature vectors using mean aggregation for storage in the VDMS vector store.
• Query Logic: Performs similarity searches in the vector store to find matching videos, which are then passed to a Large Vision Model (LVM) for contextual inference.