DOCSEARCHbyAlgolia
Submit Docs

GenAIComps Framework

repository·main·Indexed 42 days ago

https://github.com/opea-project/genaicomps

A framework for building enterprise-grade Generative AI applications using a microservice architecture. It provides modular building blocks (Microservices) that can be composed into larger applications (MegaServices) for tasks such as RAG, embedding, and LLM serving. It includes a versatile Agent Microservice supporting multiple strategies including ReAct, RAG Agent, Plan-Execute, and SQL Agent, with specialized implementations for open-source LLMs (via vLLM/TGI) and OpenAI models.

Tokens
205.2K
Snippets
658
Records
836
Agent score
52%

What's inside GenAIComps

  1. Overview of the ASR Microservice

    main
    The ASR (Audio-Speech-Recognition) microservice converts speech audio into text. This is a critical component for building conversational bots with LLMs, as it provides the text transcription required for the LLM to tokenize input and generate responses. The microservice supports multiple backend models and deployment platforms (CPU and HPU).

  2. Overview of the Text-to-SQL Microservice

    main
    The Text-to-SQL microservice uses LLMs and agentic frameworks (built on LangChain/LangGraph) to transform natural language questions into SQL queries. It executes these queries against a database to retrieve data, making it suitable for ChatBots, custom dashboards, and non-technical data retrieval.

  3. Overview of the LVM Microservice

    main
    The LVM (Large Visual Model) Microservice enables Visual Question and Answering (VQA) tasks. It allows users to interact with visual inputs (images and videos) by providing a text prompt and a visual file, receiving a text answer in return. The service abstracts the complexity of different underlying models through a standardized API, making it suitable for integration into multimodal GenAI workflows.

  4. Overview of Bias Detection Microservice

    main

    The Bias Detection Microservice is a lightweight guardrail designed for AI Application developers to safeguard user input (Pre-LLM) and LLM output (Post-LLM) from biased language in RAG environments. It uses fine-tuned Transformer models (such as DistilledBERT or RoBERTa) to classify three types of bias:

    • Framing bias: Subjective words or phrases linked to a specific point of view.
    • Epistemological bias: Linguistic features that modify the believability of a proposition.
    • Demographic bias: Text containing presuppositions about gender, race, or other demographic categories.

    The service is optimized for deployment on Intel Gaudi and Xeon hardware.

  5. Overview of Prediction Guard Microservice

    main
    The Prediction Guard microservice provides access to hosted open access LLMs, LVMs, and embedding functionality with integrated safeguards. It allows users to configure factual consistency checks, toxicity filters, PII filters, and prompt injection blocking. To use this service, you must request an API key from the Prediction Guard team via their Discord channel.

  6. Overview of the Retriever Microservice (openGauss)

    main
    The Retriever Microservice is a high-efficiency search service designed to perform similarity searches against vectors stored in an openGauss VectorDB. It accepts an embedding vector as input and retrieves the most contextually similar documents from a specified index using vector space similarity measures. This service is ideal for recommendation systems and information retrieval tasks where precise document similarity is required.

  7. Overview of the Retriever Microservice

    main

    The Retriever Microservice is a high-efficiency search service designed to handle and retrieve embedding vectors. It performs similarity searches against vectors stored in a VectorDB (specifically OpenSearch in this implementation).

    Core Workflow:

    1. Receives an embedding vector and text as input.
    2. Conducts a similarity search within a specified OpenSearch index.
    3. Returns documents with the highest similarity to the input vector.

    This service is ideal for recommendation systems, information retrieval, and applications requiring precise document similarity measurements.

  8. What is the Retriever Microservice (PGVector)

    main
    The Retriever Microservice is a search service designed to handle and retrieve embedding vectors. It performs similarity searches by receiving an embedding vector as input and querying a VectorDB (specifically PGVector in this implementation) to find documents with the highest similarity. It is suitable for recommendation systems and information retrieval tasks where fast, accurate document similarity measurement is required.

  9. Overview of the Embeddings Microservice

    main
    The Embeddings Microservice converts textual strings into vectorized embeddings for use in machine learning and data processing workflows (e.g., NLP, information retrieval). It is designed to be highly performant, scalable, and easy to integrate via a simple API. Users can customize the service by selecting different backend implementations to meet specific model or deployment requirements.

  10. Overview of the Image-to-Video Microservice

    main
    The Image-to-Video microservice is a single-purpose service designed to generate short video clips from a provided source image. It uses the Stable Video Diffusion (SVD) model to perform the generation task. The service exposes an API endpoint that accepts an image and returns the resulting video.

  11. Overview of the MariaDB Retriever Microservice

    main
    The MariaDB Retriever Microservice is a high-efficiency search service designed to handle and retrieve embedding vectors. It performs similarity searches by receiving an embedding vector as input and querying a VectorDB (MariaDB) to find documents with the highest similarity. This service is ideal for recommendation systems and information retrieval tasks where precise document similarity measurement is required.

  12. Introduction to Toxicity Detection Microservice

    main

    The Toxicity Detection Microservice is a lightweight guardrails component designed to safeguard AI applications by detecting harmful language in user inputs (Pre-LLM) or LLM outputs (Post-LLM).

    It uses a fine-tuned Transformer model (specifically Intel/toxic-prompt-roberta) to classify text for toxicity, including aggression, bullying, hate speech, or offensive language. Because it uses a Small Language Model (SLM) rather than a large decoder LLM, it offers significantly higher throughput and lower latency compared to solutions like LlamaGuard.