DOCSEARCHbyAlgolia
Submit Docs

HyperGraphRAG

repository·main·Indexed 19 days ago

https://github.com/lhrlab/hypergraphrag

A Retrieval-Augmented Generation (RAG) framework that uses hypergraph-structured knowledge representation to improve retrieval and generation quality. The library provides a HyperGraphRAG class for constructing knowledge hypergraphs via the .insert() method and performing queries via the .query() method. It includes an evaluation pipeline for knowledge construction, retrieval, response generation, and scoring across various domain classes such as hypertension, agriculture, cs, and legal.

Tokens
2K
Snippets
10
Records
11
Agent score
15%

What's inside HyperGraphRAG

  1. Run the HyperGraphRAG evaluation pipeline

    main

    The evaluation process consists of five distinct steps. You can run these sequentially to construct the hypergraph, retrieve knowledge, generate responses, and score them.

    Step 1: Knowledge HyperGraph Construction

    Construct the hypergraph for a specific class using script_insert.py. Use nohup to run this in the background.

    Step 2: Retrieve Knowledge

    Retrieve knowledge using the HyperGraphRAG method via script_hypergraphrag.py.

    Step 3: Generate Responses

    Generate answers based on the retrieved knowledge using get_generation.py.

    Step 4: Evaluate Generation

    Score the generated responses using get_score.py. This step requires a GPU.

    Step 5: View Results

    View the final evaluation scores using see_score.py.

    # Example pipeline for 'hypertension' data source

# 1. Construction
nohup python script_insert.py --cls hypertension > result_hypertension_insert.log 2>&1 &

# 2. Retrieval
python script_hypergraphrag.py --data_source hypertension

# 3. Generation
python get_generation.py --data_sources hypertension --methods HyperGraphRAG

# 4. Evaluation
CUDA_VISIBLE_DEVICES=0 python get_score.py --data_source hypertension --method HyperGraphRAG

# 5. View Results
python see_score.py --data_source hypertension --method HyperGraphRAG

  2. Prepare the HyperGraphRAG evaluation environment

    main

    To evaluate HyperGraphRAG, follow these setup steps:

    1. Working Directory: Navigate to the evaluation directory.
    2. API Configuration: Set your OpenAI API key in the openai_api_key.txt file. (The documentation notes the use of www.apiyi.com for the LLM server).
    3. Data Setup: Download the required contexts and datasets from the provided Terabox link and place them in the contexts/ and datasets/ folders respectively.

    Expected directory structure:

    HyperGraphRAG/
└── evaluation/
    ├── contexts/   
    │   ├── hypertension_contexts.json   
    │   ├── agriculture_contexts.json    
    │   ├── cs_contexts.json                  
    │   ├── legal_contexts.json                    
    │   └── mix_contexts.json    
    ├── datasets/           
    │   ├── hypertension/                             
    │   │   └── questions.json     
    │   ├── agriculture/                            
    │   │   └── questions.json 
    │   ├── cs/                            
    │   │   └── questions.json 
    │   ├── legal/                            
    │   │   └── questions.json 
    │   └── mix/                            
    │       └── questions.json
    └── openai_api_key.txt                               
    cd evaluation

  3. Install HyperGraphRAG via Conda and Pip

    main

    To set up the HyperGraphRAG environment, create a new Conda environment with Python 3.11 and install the required dependencies using pip.

    conda create -n hypergraphrag python=3.11
conda activate hypergraphrag
pip install -r requirements.txt

  4. Construct a Knowledge HyperGraph

    main

    To build a knowledge hypergraph, initialize the HyperGraphRAG class with a working_dir and use the .insert() method to ingest context data. Ensure the OPENAI_API_KEY environment variable is set.

    Note: The input to .insert() should be a list of contexts (e.g., loaded from a JSON file).

    import os
import json
from hypergraphrag import HyperGraphRAG

os.environ["OPENAI_API_KEY"] = "your_openai_api_key"

rag = HyperGraphRAG(working_dir=f"expr/example")

with open(f"example_contexts.json", mode="r") as f:
    unique_contexts = json.load(f)
    
rag.insert(unique_contexts)

  5. Query the Knowledge HyperGraph

    main

    Once the hypergraph is constructed, you can perform Retrieval-Augmented Generation queries using the .query() method. This method takes a string query and returns the generated result.

    import os
from hypergraphrag import HyperGraphRAG

os.environ["OPENAI_API_KEY"] = "your_openai_api_key"

rag = HyperGraphRAG(working_dir=f"expr/example")

query_text = 'How strong is the evidence supporting a systolic BP target of 120–129 mmHg in elderly or frail patients?'

result = rag.query(query_text)
print(result)

  6. HyperGraphRAG API Reference

    main

    The HyperGraphRAG class is the primary interface for the library.

    HyperGraphRAG(working_dir: str)

    Initializes the RAG engine.

    • working_dir: A string path specifying the directory where the hypergraph data and state will be stored.

    insert(contexts: list)

    Ingests knowledge into the hypergraph structure.

    • contexts: A list of context objects (typically loaded from JSON) used to construct the hypergraph.

    query(query_text: str)

    Performs a retrieval-augmented generation query.

    • query_text: The natural language question or prompt.
    • Returns: The generated response based on the hypergraph knowledge.

  7. Generate responses with get_generation.py

    main

    Use get_generation.py to generate answers based on retrieved knowledge.

    Arguments:

    • --data_sources: The data source to use (e.g., hypertension).
    • --methods: The RAG method to apply. Supported methods include HyperGraphRAG, StandardRAG, and NaiveGeneration.

    Example:

    python get_generation.py --data_sources hypertension --methods HyperGraphRAG

  8. Retrieve Knowledge with script_hypergraphrag.py

    main

    Use script_hypergraphrag.py to perform knowledge retrieval using the HyperGraphRAG method.

    Arguments:

    • --data_source: The name of the data source to use (e.g., hypertension).

    Example:

    python script_hypergraphrag.py --data_source hypertension

  9. Evaluate generation with get_score.py

    main

    Use get_score.py to score the generated responses. This command requires access to a GPU via CUDA_VISIBLE_DEVICES.

    Arguments:

    • --data_source: The data source to evaluate.
    • --method: The method used for generation (e.g., HyperGraphRAG, StandardRAG, or NaiveGeneration).

    Example:

    CUDA_VISIBLE_DEVICES=0 python get_score.py --data_source hypertension --method HyperGraphRAG

  10. Construct Knowledge HyperGraph with script_insert.py

    main

    Use script_insert.py to build the knowledge hypergraph for a specific domain class.

    Arguments:

    • --cls: The class/domain to process (e.g., hypertension, agriculture, cs, legal, or mix).

    Example:

    nohup python script_insert.py --cls hypertension > result_hypertension_insert.log 2>&1 &

  11. View evaluation results with see_score.py

    main

    Use see_score.py to display the evaluation results for a specific data source and method.

    Arguments:

    • --data_source: The data source to view.
    • --method: The generation method used.

    Example:

    python see_score.py --data_source hypertension --method HyperGraphRAG