Kotaemon

kotaemon is an open-source, customizable Retrieval-Augmented Generation (RAG) UI designed for chatting with documents. It serves two primary audiences:

• End Users: People who want to use a clean, minimalistic interface to perform QA on their documents using various LLM providers (OpenAI, Azure, Ollama, etc.).
• Developers: People who want to use kotaemon as a framework to build, customize, and host their own RAG pipelines.

Key capabilities include hybrid retrieval (full-text and vector), multi-modal document parsing (supporting figures and tables), advanced citations with in-browser PDF highlighting, and support for complex reasoning agents like ReAct and ReWOO.

In kotaemon, a component is the fundamental building block of a pipeline. Conceptually, a component is a step that takes an input, processes it, and returns an output (similar to a Python function).

Because components can contain other components as part of their processing logic, a pipeline is simply a nested component. This allows you to compose complex workflows by building larger components out of smaller, reusable ones.

The Information Panel in the Chat tab provides evidence, citations, and quality scores to help assess the reliability of the LLM's response.

Score Types

• Answer confidence: The confidence level reported by the LLM.
• Relevance score: The overall score between the evidence and the user question.
• Vectorstore score: Similarity score from vector embedding calculation (shows full-text search if retrieved from a full-text search DB).
• LLM relevant score: Relevancy score determined by the LLM using a specific prompt.
• Reranking score: Relevancy score from a Cohere reranking model.

Note: Generally, the quality hierarchy is: LLM relevant score > Reranking score > Vectorscore. By default, the overall relevance score is derived from the LLM relevant score.

The DocumentStore is responsible for holding and managing the Document objects produced by loaders. Currently, Kotaemon supports:

• InMemoryDocumentStore: Stores documents in the system memory.

Settings in ktem are categorized into three distinct types based on the stakeholder and use case:

1. Developer settings: Used for basic application customization (e.g., database URL, cloud configuration, logging configuration, and feature toggles). These are declared within flowsettings.py. Use these if you are deploying ktem to customers or building extensions.
2. Admin settings: Accessible via the Admin page. These allow administrators to configure low-level features, such as credentials for data sources or specific keys for LLM providers.
3. User settings: Designed for runtime users to personalize their experience, such as selecting output languages or choosing a reasoning type.

A pipeline is a class inheriting from kotaemon.base.BaseComponent. It consists of declared arguments (sub-pipelines or primitives) and a run method containing the execution logic.

Example structure:

from kotaemon.base import BaseComponent

class MyPipeline(BaseComponent):
    arg1: int
    arg2: str

    def run(self, arg3: str):
        return self.arg1 * self.arg2 + arg3

from kotaemon.base import BaseComponent


class SoSimple(BaseComponent):
    arg1: int
    arg2: str

    def run(self, arg3: str):
        return self.arg1 * self.arg2 + arg3

Vector stores are used to store and query vector embeddings for retrieval-augmented generation (RAG). Supported implementations include:

• ChromaVectorStore: A persistent vector store using Chroma.
• InMemoryVectorStore: A transient vector store that keeps embeddings in memory.

To run models locally (e.g., GGUF format from Hugging Face), set the LOCAL_MODEL variable in your .env file to the absolute path of your model file.

Hardware Tip: Ensure the model size leaves at least 2 GB of available RAM. For example, if you have 12 GB available, choose a model that uses at most 10 GB.

LOCAL_MODEL=<full path to your model file>

To restart the application or apply changes after the initial installation, simply run the OS-specific run_* script again from the scripts folder:

• Windows: run_windows.bat
• macOS: run_macos.sh
• Linux: run_linux.sh

# Example for Linux
bash run_linux.sh