DOCSEARCHbyAlgolia
Submit Docs

Ragbits Framework

repository·main·Indexed 23 days ago

https://github.com/deepsense-ai/ragbits

A modular framework for developing reliable and scalable GenAI applications. It includes a suite of packages: ragbits-core for fundamental LLM and prompt management, ragbits-document-search for RAG indexing and retrieval, ragbits-agents for autonomous agent orchestration and MCP integration, ragbits-evaluate for pipeline benchmarking, ragbits-guardrails for safety verification, and ragbits-chat for conversational AI interfaces.

Tokens
101.6K
Snippets
177
Records
483
Agent score
81%

What's inside Ragbits

  1. Overview of Ragbits packages

    main

    Ragbits is a modular ecosystem for building GenAI applications. While installing ragbits provides a starter bundle, you can also install individual components to reduce dependencies:

    • ragbits-core: Fundamental tools for prompts, LLMs, and vector databases.
    • ragbits-agents: Abstractions for building agentic systems.
    • ragbits-document-search: Retrieval and ingestion pipelines for knowledge bases.
    • ragbits-evaluate: Unified evaluation framework for Ragbits components.
    • ragbits-guardrails: Utilities for ensuring response safety and relevance.
    • ragbits-chat: Full-stack infrastructure for conversational AI applications.
    • ragbits-cli: The ragbits shell command for interacting with components.

  2. Overview of Ragbits Core

    main

    Ragbits Core is the foundational layer for the Ragbits ecosystem. It provides essential utilities and abstractions used by other packages, including:

    • Logging and Configuration utilities.
    • Prompt Management: Structured prompt creation using templates and Pydantic.
    • LLM Communication: Classes for interacting with various Large Language Models.
    • Embedders and Vector Stores: Tools for handling embeddings and vector database operations.

  3. Overview of Ragbits Chat

    main
    ragbits-chat is a Python package designed for building conversational AI applications. It provides a comprehensive toolkit including a framework for constructing chat experiences, mechanisms for managing conversation history to track user interactions, and UI components to facilitate the building of chat interfaces.

  4. Trace code execution with Ragbits

    main

    Ragbits includes built-in tracing for all components, providing visibility into execution flow and performance. By default, the SDK automatically traces:

    • LLM generation and streaming
    • Embedder calls (text and image)
    • Sources data fetching
    • Vector Store operations (retrieve, store, remove, list)
    • Document Search operations (search, ingest)

  5. Supported metric types in Ragbits

    main

    Ragbits supports three fundamental metric types for monitoring:

    1. Histogram metrics: Used for tracking the distribution of values, such as durations or sizes.
    2. Counter metrics: Used for tracking the count of events, such as the number of requests or errors.
    3. Gauge metrics: Used for tracking current values that can fluctuate up or down, such as memory usage.

  6. Define a Prompt using the Prompt class

    main

    The recommended way to define a prompt is to create a class that inherits from ragbits.core.prompt.Prompt. You should define an input schema using a Pydantic BaseModel and specify the system_prompt and user_prompt (using Jinja-style {{ variable }} syntax).

    from pydantic import BaseModel
from ragbits.core.prompt import Prompt

class QuestionAnswerPromptInput(BaseModel):
    question: str

class QuestionAnswerPrompt(Prompt[QuestionAnswerPromptInput, str]):
    system_prompt = """
    You are a question answering agent. Answer the question to the best of your ability.
    """
    user_prompt = """
    Question: {{ question }}
    """

  7. Choose an ingestion strategy for document search

    main

    Ragbits provides three built-in ingestion strategies for the DocumentSearch class to handle different workloads. You can select a strategy by passing an instance of it to the ingest_strategy parameter when initializing DocumentSearch.

    1. SequentialIngestStrategy: The default strategy. It processes documents one by one, waiting for each to complete before starting the next. Best for simple workloads or low-volume ingestion.
    2. BatchedIngestStrategy: Uses Python's asyncio to process documents concurrently. Use this to increase speed for large document volumes. You can control concurrency using the batch_size parameter.
    3. RayDistributedIngestStrategy: Designed for high-performance, large-scale workloads.
      • Local Mode: When run outside a Ray cluster, it parallelizes processing across available CPU cores on the local machine.
      • Cluster Mode: When run inside a Ray cluster, it parallelizes processing across multiple nodes. It is highly recommended to use the Ray Jobs API for cluster submission.
    # Example: Using BatchedIngestStrategy
from ragbits.document_search import DocumentSearch
from ragbits.document_search.ingestion.strategies import BatchedIngestStrategy

ingest_strategy = BatchedIngestStrategy(batch_size=10)
document_search = DocumentSearch(ingest_strategy=ingest_strategy, ...)

await document_search.ingest("s3://")

  8. Configure the Ragbits Chat UI Provider Hierarchy

    main

    The UI relies on a specific nesting of providers to function correctly. The hierarchy should be:

    1. HeroUIProvider: Handles component library theming and accessibility.
    2. RagbitsContextProvider: Configures the Ragbits API client with your baseUrl.
    3. ThemeContextProvider: Manages light/dark mode and localStorage persistence.
    4. HistoryStoreContextProvider: Manages the chat history and conversation state.

  9. Choose an IngestStrategy for document processing

    main

    In ragbits.document_search, ingestion strategies define how documents are processed and loaded into the system. You can choose between different implementations based on your scale and infrastructure requirements:

    • SequentialIngestStrategy: Processes documents one after another in a single sequence. Best for small datasets or environments with limited resources.
    • BatchedIngestStrategy: Processes documents in groups (batches). This is more efficient than sequential processing for medium-sized datasets as it can optimize I/O and compute.
    • RayDistributedIngestStrategy: Uses Ray to distribute the ingestion workload across a cluster. This is the recommended strategy for large-scale, high-throughput production workloads requiring distributed computing.

  10. Use the ChatInterface for chat services

    main

    The ChatInterface is the primary interface for implementing or interacting with a chat service. It is designed to handle various response types, specifically supporting:

    • Text: Regular text responses that are streamed chunk by chunk.
    • References: Source documents or citations used to generate the response.

    When building a chat service, implementing this interface ensures compatibility with the expected response patterns for streaming and document retrieval.

  11. How the Ragbits UI plugin system works

    main

    The Ragbits UI uses a plugin architecture that allows developers to extend the interface without modifying the core codebase. Plugins can perform four main actions:

    1. Register Slots: Inject components into predefined UI locations (e.g., sidebars, headers).
    2. Inject Routes: Add new pages or nested routes to the application.
    3. Wrap Routes: Use Higher-Order Components (HOCs) to wrap existing routes for logic like authentication or error boundaries.
    4. Lifecycle Hooks: Execute code when a plugin is activated or deactivated.

    The workflow involves two steps: pluginManager.register(myPlugin) to make the plugin known to the system, and pluginManager.activate(pluginName) to enable its features.

    // Example of the registration/activation flow
import { pluginManager } from "./core/utils/plugins/PluginManager";
import { MyPlugin } from "./plugins/MyPlugin";

pluginManager.register(MyPlugin);
pluginManager.activate(MyPlugin.name);
    import { pluginManager } from "./core/utils/plugins/PluginManager";
import { MyPlugin } from "./plugins/MyPlugin";

pluginManager.register(MyPlugin);
pluginManager.activate(MyPlugin.name);

  12. Prerequisites for running evaluations

    main

    Before you can run an evaluation in Ragbits, you must implement and define the following three components:

    1. EvaluationPipeline structure class: Defines the structure of the pipeline being tested.
    2. Metrics and MetricSet: Defines the specific metrics to be used and organizes them into a set.
    3. DataLoader: Defines how the input data (e.g., queries, documents) is loaded into the evaluation process.