Neo4j Knowledge Graph Builder

repository·main·Indexed 26 days ago

https://github.com/neo4j-labs/llm-graph-builder

A tool that transforms unstructured data—including PDFs, YouTube videos, and web pages—into structured Knowledge Graphs stored in Neo4j using LLMs and LangChain. It features a FastAPI backend and a ReactJS frontend, supporting deployment via Docker, Docker-Compose, and Google Cloud Platform. Requires Neo4j version 5.23 or later with APOC installed.

Tokens
31K
Snippets
87
Records
157
Agent score
89%

What's inside llm-graph-builder

  1. Prerequisites for Knowledge Graph Builder

    main

    Before installing, ensure your environment meets these requirements:

    • Python: Version 3.12 or higher (required for local/separate backend deployment).
    • Neo4j Database: Version 5.23 or later with APOC installed.
      • Note: Version 5.23 is mandatory because the backend uses Cypher variable-scope subquery syntax (CALL (variable) { ... }).
      • Neo4j Aura (including free tier) is supported.
    • Neo4j Desktop: If using Desktop, you must deploy the backend and frontend separately; docker-compose is not supported for Desktop users.
  2. Deploy using Docker-Compose

    main

    Run the application using the default docker-compose configuration. You can customize supported LLM models, input sources, and chat modes via environment variables.

    Configure LLM Models: Use VITE_LLM_MODELS_PROD to enable specific models (e.g., gemini_3.5_flash,openai_gpt_5.4_mini).

    Configure Input Sources: Use VITE_REACT_APP_SOURCES to enable sources like local, youtube, wiki, s3, gcs, or web. To use GCS, also provide VITE_GOOGLE_CLIENT_ID.

    Configure Chat Modes: Use VITE_CHAT_MODES to specify modes such as vector, graph, fulltext, etc.

  3. Set up the Backend locally

    main

    To run the backend independently of the frontend:

    1. Create a .env file in the backend folder by copying backend/example.env.
    2. Pre-configure user credentials in the .env file to bypass the login dialog:
      • NEO4J_URI
      • NEO4J_USERNAME
      • NEO4J_PASSWORD
      • NEO4J_DATABASE
    3. Initialize the virtual environment and start the server using uvicorn.
    cd backend
    python3.12 -m venv venv
    source venv/bin/activate  # On Windows: venv\Scripts\activate
    pip install -r requirements.txt
    uvicorn score:app --reload
  4. Connect to a Neo4j Aura Instance

    main

    Connect the application to a Neo4j Aura instance using the POST /connect API. You must provide the connection details in the connection modal.

    API Parameters:

    • uri: Neo4j URI
    • userName: Neo4j db username
    • password: Neo4j db password
    • database: Neo4j database name

    To verify the backend connection is active, use the GET /health endpoint. The application supports both AURA DS (Graph Data Science) and AURA DB instances. If an AURA DS connection is detected, users gain access to post-processing jobs like community detection via the 'scientific molecule' icon.

  5. Deploy to Google Cloud Platform (GCP)

    main

    Deploy the frontend and backend to Google Cloud Run using the gcloud CLI.

    # Frontend Deployment
    gcloud run deploy dev-frontend \
      --source . \
      --region us-central1 \
      --allow-unauthenticated
    
    # Backend Deployment
    gcloud run deploy dev-backend \
      --set-env-vars "OPENAI_API_KEY=<your-openai-api-key>" \
      --set-env-vars "DIFFBOT_API_KEY=<your-diffbot-api-key>" \
      --set-env-vars "NEO4J_URI=<your-neo4j-uri>" \
      --set-env-vars "NEO4J_USERNAME=<your-username>" \
      --set-env-vars "NEO4J_PASSWORD=<your-password>" \
      --source . \
      --region us-central1 \
      --allow-unauthenticated
  6. Install and set up the backend locally

    main

    To run the backend project locally, ensure you have Python 3.12 or higher and pip installed. Follow these steps:

    1. Clone the repository:
      git clone https://github.com/neo4j-labs/llm-graph-builder.git
      cd llm-graph-builder
    2. Create and activate a virtual environment:
      python3.12 -m venv venv
      source venv/bin/activate  # On Windows: venv\Scripts\activate
    3. Install dependencies:
      pip install -r requirements.txt
    python3.12 -m venv venv
    source venv/bin/activate
    pip install -r requirements.txt
  7. Set up the Frontend locally

    main

    To run the frontend independently of the backend:

    1. Create a .env file in the frontend folder by copying frontend/example.env.
    2. Update environment variables as needed.
    3. Install dependencies with yarn and start the development server.
    cd frontend
    yarn
    yarn run dev
  8. Visualize the Extracted Graph

    main

    View the generated graph using the POST /graph_query API. The application supports three view modes:

    1. Lexical Graph: Document and Chunk relationships.
    2. Entity Graph: Extracted entities and their relationships.
    3. Knowledge Graph: High-level knowledge representation.

    Features:

    • Customization: Zoom, fit, refresh, node styling, and relationship type filtering.
    • Neighborhood View: Use POST /get_neighbours with an elementId to retrieve the neighbors of a specific node.
  9. Deploy the Frontend to Google Cloud Run

    main

    Use the gcloud CLI to deploy the frontend service to Google Cloud Run. When prompted, select the current directory for the source location and choose a region (e.g., us-central1). Set 'Allow unauthenticated requests' to Yes.

    gcloud run deploy 
    source location current directory > Frontend
    region : 32 [us-central 1]
    Allow unauthenticated request : Yes
  10. Configure Graph Schema and Processing

    main

    Customize how the graph is built and maintained:

    • Schema Management:
      • POST /schema: Fetch existing schema.
      • POST /populate_graph_schema: Generate a schema from provided input text.
    • Embedding Models: Use POST /fetch_embedding_model to see available providers and POST /change_embedding_model to switch them.
    • Graph Maintenance:
      • POST /delete_unconnected_nodes: Remove orphan entities.
      • POST /get_unconnected_nodes_list: List orphan nodes.
      • POST /get_duplicate_nodes: Find duplicate entities.
      • POST /merge_duplicate_nodes: Merge selected duplicate entities.
      • POST /drop_create_vector_index: Rebuild vector indexes if dimensions change.
    • Post-Processing: Use POST /post_processing to fine-tune the graph (e.g., creating KNN relations between similar chunks).