neuml/rag

repository·master·Indexed 19 days ago

A Streamlit-based Retrieval Augmented Generation (RAG) application powered by txtai. It supports traditional Vector RAG and advanced Graph RAG, including graph query expansion and path traversal. The application allows for dynamic data ingestion via the '#' prefix and supports multiple LLM backends including GPT-OSS, Ollama, and GPT-5.1.

Tokens: 1.7K
Snippets: 6
Records: 7
Agent score: 16%

What's inside neuml/rag

How Vector RAG and Graph RAG work
master
The application supports two primary RAG methodologies:
Vector RAG
Traditional RAG that performs a vector search to find the top $N$ most relevant matches to a user's input. These matches are then used to hydrate an LLM prompt.
Graph RAG
Uses knowledge or semantic graphs to generate context via graph path traversal. It supports three query patterns:
1. Graph query expansion: Prefix a query with gq: to perform a vector search followed by expansion using a graph network.
  Example: gq: Tell me about Linux
2. Graph path query: Provide a list of concepts separated by -> to find related nodes via traversal.
  Example: linux -> macos -> microsoft windows
3. Combination: Run a graph path query followed by a graph query within that path's context.
  Example: linux -> macos -> microsoft windows gq: Tell me about Linux
Every Graph RAG response includes a visual graph where nodes represent sections (paragraphs) labeled with topics.
https://github.com/neuml/rag/blob/master/README.md

Configure Embeddings and Data Indexing

master

Use the EMBEDDINGS variable to specify a different index or start with an empty one. To build an index from a local directory, use the DATA variable and mount the volume.

Use a specific embeddings index

docker run -d --gpus=all -it -p 8501:8501 -e EMBEDDINGS=neuml/arxiv neuml/rag

Start with an empty embeddings index

docker run -d --gpus=all -it -p 8501:8501 -e EMBEDDINGS= neuml/rag

Build index from a local directory

docker run -d --gpus=all -it -p 8501:8501 -e DATA=/data/path -v local/path:/data/path neuml/rag

Persist embeddings and cache models

To ensure data and models are not lost when the container stops, map volumes for DATA, EMBEDDINGS, PERSIST, and HF_HOME.

docker run -d --gpus=all -it -p 8501:8501 -e DATA=/data/path -e EMBEDDINGS=/data/embeddings -e PERSIST=/data/embeddings -e HF_HOME=/data/modelcache -v localdata:/data neuml/rag

# Build an embeddings index with a local directory of files
docker run -d --gpus=all -it -p 8501:8501 -e DATA=/data/path -v local/path:/data/path neuml/rag

Install and run neuml/rag via Python virtual environment
master
You can install the application directly into a Python environment. It is recommended to use a virtual environment.
1. Install dependencies:
```
pip install -r requirements.txt
```
1. Start the Streamlit application:
```
streamlit run rag.py
```
```
pip install -r requirements.txt
streamlit run rag.py
```
https://github.com/neuml/rag/blob/master/README.md
Install and run neuml/rag via Docker
master
The recommended way to run the application is using Docker. You can run it with default settings using the following command. This command includes GPU support and maps port 8501 for the Streamlit interface.
```
docker run -d --gpus=all -it -p 8501:8501 neuml/rag
```
https://github.com/neuml/rag/blob/master/README.md
Add data to the index using the '#' prefix
master
You can dynamically add data to the embeddings index by starting your query with a # symbol. This allows you to ingest files, URLs, or raw text directly into the RAG system.
- File or URL: # path/to/file.pdf or # https://example.com/doc.txt
- Custom Text: # your custom text here (this creates a new entry in the Embeddings database).
```
# https://example.com/data.txt
# txtai is an all-in-one AI framework
```
https://github.com/neuml/rag/blob/master/README.md

Configure LLM providers (GPT-OSS, Ollama, GPT-5.1)

master

You can switch between different LLM backends by setting the LLM environment variable.

GPT-OSS

docker run -d --gpus=all -it -p 8501:8501 -e LLM=openai/gpt-oss-20b neuml/rag

GPT-OSS via Ollama

To use a local Ollama instance, you must map the host gateway to allow the container to reach your host machine.

docker run -d --gpus=all -it -p 8501:8501 --add-host=host.docker.internal:host-gateway -e LLM=ollama/gpt-oss -e OLLAMA_API_BASE=http://host.docker.internal:11434 neuml/rag

GPT-5.1

docker run -d --gpus=all -it -p 8501:8501 -e LLM=gpt-5.1 -e OPENAI_API_KEY=your-api-key neuml/rag

# GPT-OSS via Ollama example
docker run -d --gpus=all -it -p 8501:8501 --add-host=host.docker.internal:host-gateway -e LLM=ollama/gpt-oss -e OLLAMA_API_BASE=http://host.docker.internal:11434 neuml/rag

Configure the RAG application via environment variables

master

The application behavior is controlled by several environment variables. When running via Docker, use the -e flag; when running in a Python environment, set them in your shell or .env file. You can view current settings in the app by typing :settings.

| Variable | Description | Default Value |
|:----------- |:------------------------------------------- |:----------------------------------- |
| TITLE | Main title of the application | 🚀 RAG with txtai |
| EXAMPLES | List of queries separated by `;` | `Who created Linux?; gq: Tell me about Linux; linux -> macos -> microsoft windows; linux -> macos -> microsoft windows gq: Tell me about Linux` |
| LLM | Path to LLM | Qwen3-4B-Instruct-2507 |
| EMBEDDINGS | Embeddings database path | neuml/txtai-wikipedia-slim |
| MAXLENGTH | Maximum generation length | 2048 for topics, 4096 for RAG |
| STRIPTHINK | Strip thinking text from responses | False |
| CONTEXT | RAG context size | 10 |
| TEXTBACKEND | Text extraction backend | available |
| DATA | Optional directory to index data from | None |
| PERSIST | Optional directory to save index updates to | None |
| TOPICSBATCH | Optional batch size for LLM topic queries | None |
| SAFEOPEN | Enable Textractor safeopen mode | True |