RAGFlow

The es-ob-migrate tool is a CLI utility designed to migrate RAGFlow data from an Elasticsearch (ES) backend to an OceanBase backend. It is purpose-built for RAGFlow's specific data schema and includes features for:

• Schema Conversion: Handles RAGFlow's fixed data structure and vector field dimension auto-detection.
• Efficient Data Transfer: Uses the ES search_after API for high-performance scrolling.
• Reliability: Supports batch processing, resume capability for interrupted migrations, and data consistency validation.
• Reporting: Generates detailed migration and verification reports.

RAGFlow's Team Management system allows you to collaborate by sharing resources among multiple users. By default, every user is assigned a personal team named after them. However, you can create or join teams to enable collaborative workflows.

Collaborative Capabilities: When users join a shared team, they can collectively:

• Upload documents to shared datasets.
• Parse documents within those shared datasets.
• Utilize shared Agents.

Permissions Model:

• Team Owner: The only role with authority to invite new users to the team or remove existing members.
• Team Members: Can contribute to datasets and agents but lack administrative privileges (cannot invite or remove users).

Users interact with workspaces through a Team page, which is organized into two primary sections:

1. Team members: Displays all users currently belonging to the active workspace.
2. Joined teams: Lists other workspaces that the user has either joined independently or has been invited to join.

Retrieval-Augmented Generation (RAG) is a technique used to solve the limitations of Large Language Models (LLMs), such as static knowledge, inability to access private data, and hallucinations.

Instead of relying solely on the LLM's internal training data, RAG follows a two-stage process:

1. Retrieval Stage: The system searches an external knowledge base to find the most relevant documents or data fragments based on the user's query.
2. Generation Stage: The retrieved information is provided to the LLM as "reference material" (context), which the LLM then uses to generate an accurate, grounded answer.

This transforms the LLM from "speaking from memory" to "speaking with documentation."

RAPTOR (Recursive Abstractive Processing for Tree Organized Retrieval) is an enhanced document preprocessing technique that builds a hierarchical tree structure of document chunks. It works by recursively clustering chunks based on semantic similarity and summarizing those clusters using a chat model.

When to use it: Use RAPTOR for multi-hop question-answering tasks that require complex, multi-step reasoning. It helps bridge the semantic gap between a question and its answer by providing the model with both high-level summaries (root nodes) and granular details (leaf nodes).

Warning: Enabling RAPTOR requires significant memory, computational resources, and tokens because it performs recursive summarization via your chat model.

DeepDoc is designed for high-quality document analysis and consists of two main components:

1. Visual Processing: Includes OCR for text extraction, Layout Recognition for identifying document structures (like headers, figures, and tables), and Table Structure Recognition (TSR) for parsing complex tables into machine-readable formats.
2. Parser: Provides specialized parsers for PDF, DOCX, EXCEL, and PPT. The PDF parser is the most advanced, outputting text fragments with positional data, cropped images, and tables converted into natural language sentences.

Cross-language search (cross-lingual retrieval) allows users to submit queries in one language (e.g., English) and retrieve relevant documents written in different languages (e.g., Chinese or Spanish).

This is achieved by the system's default chat model, which translates queries to match semantic meaning across languages. This feature helps overcome language barriers and increases the scope of searchable information. It was introduced in version 0.19.0.

The RAGFlow Sandbox is designed to support multiple deployment modes to allow for different execution environments. Currently, it supports:

• Self-Managed: On-premise deployment utilizing Daytona or Docker.
• SaaS providers: Cloud-based sandbox services such as Aliyun Code Interpreter or E2B.

The architecture is built around a provider-agnostic interface, allowing RAGFlow to interact with different sandbox backends through a unified set of operations while maintaining multi-tenant isolation (mapping one session to one sandbox) and providing unified monitoring.

The Admin Service is the core backend management service of RAGFlow. It uses a client-server architecture to provide centralized API interfaces for system administration. It can be controlled via a Web UI or the Admin CLI.

Core Capabilities:

• System Monitoring: Real-time monitoring of the RAGFlow server and critical dependencies including MySQL, Elasticsearch, Redis, and MinIO.
• User Management: In administrator mode, you can view user information, create users, update passwords, modify activation status, and delete user data.
• Resilience: Administrative functions remain accessible via the Admin CLI even if the Web UI is disabled.

An Agent is RAGFlow's business workflow orchestration capability. It allows users to build complex logic using a no-code canvas by adding components and defining their execution order through connections.

Agents support various execution patterns:

• Sequential execution: Components run one after another.
• Conditional branching: Paths change based on classification results or specific conditions.
• Loop logic: Repeating certain steps within the workflow.

Common use cases include:

• Answering questions using knowledge bases.
• Intent identification and routing to specific processing flows.
• Interacting with external systems via HTTP interfaces, databases, MCP tools, or custom code.
• Processing long texts through splitting and batching.
• Managing session memory or exporting results.