HippoRAG Documentation

You can use HippoRAG with a locally deployed vLLM server.

1. Start vLLM server: Set VLLM_WORKER_MULTIPROC_METHOD=spawn to avoid CUDA initialization issues when using multiple GPUs.

   export CUDA_VISIBLE_DEVICES=0,1
   export VLLM_WORKER_MULTIPROC_METHOD=spawn
   export HF_HOME=<path>
   vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 2 --max_model_len 4096 --gpu-memory-utilization 0.95

2. Connect HippoRAG: Point llm_base_url to your vLLM server (e.g., http://localhost:8000/v1).

To use HippoRAG with OpenAI, set your OPENAI_API_KEY and initialize the HippoRAG class with your desired model names. The workflow consists of calling .index(docs=...) followed by .rag_qa(queries=...).

Note: For OpenAI-compatible endpoints (like local servers), you can provide llm_base_url and embedding_base_url.

from hipporag import HippoRAG

docs = ["George Rankin is a politician."]
queries = ["What is George Rankin's occupation?"]
hipporag = HippoRAG(save_dir="outputs", llm_model_name="gpt-4o-mini", embedding_model_name="text-embedding-3-small")
hipporag.index(docs=docs)
results = hipporag.rag_qa(queries=queries)

HippoRAG supports Amazon Bedrock via LiteLLM.

1. Standard Bedrock Runtime: Prefix the model ID with bedrock/ (e.g., bedrock/model-id).
2. Amazon Bedrock Mantle: For models like GPT-5.5, use the Mantle endpoint. You must set AWS_BEARER_TOKEN_BEDROCK and provide the specific llm_base_url.

To use an existing AWS profile instead of a bearer token, configure BaseConfig with:

• bedrock_mantle_auth='aws_credentials'
• bedrock_aws_profile='<profile>'
• bedrock_region='<region>'

hipporag = HippoRAG(
    save_dir='outputs/bedrock-mantle',
    llm_model_name='bedrock-mantle/openai.gpt-5.5',
    llm_base_url='https://bedrock-mantle.us-east-2.api.aws/openai/v1',
    embedding_model_name=embedding_model_name,
)

Azure OpenAI is supported via the main.py entry point. You must export AZURE_OPENAI_API_KEY and provide the --azure_endpoint (chat completions URL) and --azure_embedding_endpoint (embeddings URL) as CLI arguments.

export AZURE_OPENAI_API_KEY=<your Azure OpenAI API key>
python main.py --dataset sample --embedding_name text-embedding-3-small --azure_endpoint <chat-completions-url> --azure_embedding_endpoint <embeddings-url>

You can install HippoRAG using Conda or uv. A Python 3.10 environment is required.

Using Conda:

conda create -n hipporag python=3.10
conda activate hipporag
pip install hipporag

Using uv:

uv venv --python 3.10 .venv
source .venv/bin/activate
uv pip install -e .

Required Environment Variables: Set only the variables required by your chosen models:

• CUDA_VISIBLE_DEVICES: Specify GPU IDs.
• HF_HOME: Path to Huggingface home directory.
• OPENAI_API_KEY: Required if using OpenAI models.

For faster indexing (potentially >3x faster than the online server), use vLLM's offline batch mode. This mode sets VLLM_WORKER_MULTIPROC_METHOD=spawn internally.

1. Run the main program with --openie_mode offline to generate OpenIE results.
2. Once OpenIE results are saved, switch to the standard vLLM online server method to complete the process.

export CUDA_VISIBLE_DEVICES=0,1,2,3
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export HF_HOME=<path to Huggingface home directory>
export OPENAI_API_KEY=''
dataset=sample

python main.py --dataset $dataset --llm_name meta-llama/Llama-3.3-70B-Instruct --openie_mode offline

To run HippoRAG using OpenAI models, ensure you have initialized your environment variables (OPENAI_API_KEY, HF_HOME, and CUDA_VISIBLE_DEVICES) and activated the hipporag conda environment. Use the main.py script specifying the dataset, the OpenAI base URL, the model name, and the embedding model.

To run a standard dense-retrieval/DPR-style baseline instead of HippoRAG, add the --rag_type standard flag.

By default, HippoRAG uses local Parquet files. To use Milvus, first install the dependency: pip install "hipporag[milvus]".

Milvus Lite: Uses a local database inside the HippoRAG working directory. No server required.

from hipporag import HippoRAG
from hipporag.utils.config_utils import BaseConfig

config = BaseConfig(vector_store_type="milvus")
# Pass config to HippoRAG via global_config

Milvus Server / Zilliz Cloud: Set milvus_uri and milvus_token in BaseConfig, or use environment variables:

• MILVUS_URI
• MILVUS_TOKEN
• MILVUS_DB_NAME
• MILVUS_CONSISTENCY_LEVEL

config = BaseConfig(
    vector_store_type="milvus",
    milvus_uri="http://localhost:19530",
    milvus_token=None,
    milvus_db_name=None,
    milvus_consistency_level="Session",
)

You can use ChromaDB as the embedding backend for HippoRAG in two modes: local (persistent file) or remote (HTTP server).

To use a local instance, provide a BaseConfig with vector_store_type="chroma". The data will be stored in a chroma_storage directory within your specified db_path.

To use a remote instance, provide chroma_host and chroma_port in your BaseConfig.

# Local — no server needed
HippoRAG(global_config=BaseConfig(vector_store_type="chroma"))

# Remote
HippoRAG(global_config=BaseConfig(vector_store_type="chroma",
                                 chroma_host="localhost", chroma_port=8000))

To use Amazon Bedrock models in HippoRAG, initialize the system by providing a valid Bedrock Model-ID to the llm_model_name configuration parameter. This implementation uses litellm under the hood to interface with Bedrock.

Example Model-ID: anthropic.claude-3-5-haiku-20241022-v1:0

To use a HuggingFace Transformers model as the LLM backend in HippoRAG, set the llm_name in your configuration to include the Transformers/ prefix followed by the HuggingFace Model-ID (e.g., Transformers/meta-llama/Llama-3.1-8B-Instruct).

TransformersLLM automatically handles model loading via AutoModelForCausalLM with device_map="auto" and torch_dtype=torch.bfloat16. It also implements an automatic SQLite-based caching mechanism to save LLM responses, stored in a subdirectory named llm_cache within your save_dir.