ROGRAG: Robustly Optimized GraphRAG

ROGRAG is a robust GraphRAG (Graph Retrieval-Augmented Generation) approach designed to enhance LLM performance on specialized topics. It utilizes a two-stage retrieval mechanism (dual-level and logic form methods) to improve accuracy without increasing computational costs.

Key features include:

• Two-stage retrieval: Enhances robustness in finding relevant information.
• Incremental database construction: Allows for efficient knowledge updates.
• Enhanced fuzzy matching and structured reasoning: Improves the precision of retrieved knowledge.

ROGRAG has demonstrated significant performance improvements on the SeedBench benchmark compared to vanilla RAG, LangChain, BM25, and RQ-RAG.

ROGRAG is a robustly optimized GraphRAG implementation that merges multiple open-source projects (HuixiangDou, KAG, LightRAG, and DB-GPT) to improve retrieval accuracy. It is designed to overcome limitations in standard RAG pipelines by using a two-stage retrieval process that strengthens fuzzy matching and logical reasoning retrieval. It also supports incremental knowledge base construction.

Key features include:

• Two-stage retrieval: Enhances fuzzy matching and logical reasoning.
• Incremental knowledge base construction: Allows for efficient updates to the graph.
• High accuracy: Demonstrated significant improvements over vanilla RAG and LangChain on specialized test sets.

The project is organized into several top-level directories that separate implementation, testing, and configuration:

• huixiangdou/: The core implementation of the system.
• config.ini.example: A configuration template (note: this file uses TOML format despite the .ini extension).
• evaluation/: Tools for testing pipeline accuracy.
• tests/: Verification of specific code snippets.
• unittest/: Standard unit tests for the codebase.

.
├── config.ini.example  # Configuration example
├── evaluation          # Pipeline accuracy testing tool
├── huixiangdou         # Implementation
├── tests               # Code snippet verification
└── unittest            # Unit tests

The project is organized into several top-level directories that separate configuration, implementation, testing, and evaluation:

• config.ini.example: A template configuration file (note: the actual format is TOML, despite the .ini extension).
• huixiangdou/: The core implementation of the project.
• evaluation/: Tools for testing pipeline accuracy.
• tests/: Verification of specific code snippets.
• unittest/: Standard unit tests.

.
├── config.ini.example  # 配置样例
├── evaluation          # pipeline 精度测试工具
├── huixiangdou         # 实现
├── tests               # 代码片段验证
└── unittest            # 单元测试

When setting up your RAG pipeline, the choice of splitter affects the quality of the retrieved chunks. Based on evaluation results:

• For Chinese priority: Use ChineseTextSplitter. This results in centrifugal values.
• For English priority: Use langchain.RecursiveTextSplitter. This cuts Chinese corpus more finely but lacks centrifugal values.
• Avoid: CharacterTextSplitter, as it does not actually slice the text effectively and should not be used for direct slicing.

When choosing a splitter for text processing, consider the following trade-offs observed during testing:

• ChineseTextSplitter: Prioritizes Chinese text but may result in outlier values (离心值).
• langchain.RecursiveTextSplitter: Good for English; when used on Chinese corpora, it results in smaller/more fragmented chunks but avoids outlier values.
• CharacterTextSplitter: Does not effectively perform splitting in practice; avoid using it directly.

To quickly familiarize yourself with HuixiangDou, follow these steps in order:

1. Run the basic version: Follow the instructions in the main README.md to get the system up and running with default settings.
2. Enhance performance: Refer to the advanced tutorial to optimize and enhance the overall effect of the system.

For a more detailed walkthrough, consult the doc_how_to_run.md guide in the Quick Start section.

Before running the pipeline, you must configure the config.ini file. This includes setting your LLM API key (e.g., from SiliconCloud).

1. Copy the example configuration file to create your active config:

   cp config.ini.example config.ini

2. Open config.ini and fill in your SiliconCloud SK (API Key).

For a detailed breakdown of all configuration options, refer to the Configuration Guide.

To perform rejection accuracy testing (evaluating the upper and lower bounds of chunksize for text2vec models), follow these steps:

1. Prepare Data: Populate gt_bad.txt and gt_good.txt with your annotated positive and negative examples.
2. Execute Processing: Run the filtering script to calculate tokenized lengths in debug mode:

   python3 evaluation/rejection/build_fs_and_filter.py

3. Visualize Results: Use the plotting script to generate F1 score charts for different chunksize and throttle values:

   python3 plot.py

After starting VLLM, you can build the knowledge base using documents located in HuixiangDou/tests/data.

1. Copy the test data to the repodir directory:

   cd HuixiangDou
   cp -rf tests/data repodir

2. Run the pipeline store module:

   python3 -m huixiangdou.pipeline.store

Once built, you can view the resulting graph by opening port 17070 in a browser. Use credentials admin / 73@TuGraph to log in.