WhyHow Knowledge Graph Studio

repository·main·Indexed 19 days ago

https://github.com/whyhow-ai/knowledge-graph-studio

A platform for creating and managing RAG-native knowledge graphs featuring rule-based entity resolution, modular graph construction, and a NoSQL (MongoDB) backend. It includes a Python SDK (whyhow-api) for programmatic interaction, an admin CLI for database and user management, and a set of API routers for managing workspaces, schemas, graphs, triples, nodes, and documents.

Tokens
15.2K
Snippets
56
Records
69
Agent score
75%

What's inside WhyHow Knowledge Graph Studio

  1. Create a User and API Key

    main

    After configuring collections, you must create a user to obtain an API Key. This key is required to communicate with the backend via the SDK.

    Run the following from src/whyhow_api/cli/:

    $ python admin.py create-user --email <your email address> --openai-key <your openai api key>

    Important: Copy the API Key provided in the output message immediately, as you will need it to configure the WhyHow client.

  2. Use the WhyHow Python SDK

    main

    To interact with the Knowledge Graph Studio programmatically, install the whyhow SDK and use the WhyHow client.

    Installation:

    $ pip install whyhow

    Basic Workflow:

    1. Initialize Client: Pass your api_key and the base_url of your running server.
    2. Create Workspace: Organize your data within a workspace.
    3. Create Chunks: Ingest text content as Chunk objects.
    4. Create Triples: Define relationships between Node objects (head and tail) using a Relation.
    5. Build Graph: Use create_graph_from_triples to materialize the graph.
    6. Query: Use query_unstructured to perform natural language queries against the graph.
    from whyhow import WhyHow, Triple, Node, Chunk, Relation
    
    # Configure WhyHow client
    client = WhyHow(api_key='<your whyhow api key>', base_url="http://localhost:8000")
    
    # Create workspace
    workspace = client.workspaces.create(name="Demo Workspace")
    
    # Create chunk(s)
    chunk = client.chunks.create(
        workspace_id=workspace.workspace_id,
        chunks=[Chunk(
            content="preneur and visionary, Sam Altman serves as the CEO of OpenAI, leading advancements in artific"
        )]
    )
    
    # Create triple(s)
    triples = [
        Triple(
            head=Node(
                name="Sam Altman",
                label="Person",
                properties={"title": "CEO"}
            ),
            relation=Relation(
                name="runs",
            ),
            tail=Node(
                name="OpenAI",
                label="Business",
                properties={"market cap": "$157 Billion"}
            ),
            chunk_ids=[c.chunk_id for c in chunk]
        )
    ]
    
    # Create graph
    graph = client.graphs.create_graph_from_triples(
        name="Demo Graph",
        workspace_id=workspace.workspace_id,
        triples=triples
    )
    
    # Query graph
    query = client.graphs.query_unstructured(
        graph_id=graph.graph_id,
        query="Who runs OpenAI?"
    )
  3. Initialize MongoDB Collections and Indexes

    main

    Use the provided admin script to create the necessary database collections and indexes in your MongoDB Atlas cluster. This script creates 11 collections: chunk, document, graph, node, query, rule, schema, task, triple, user, and workspace.

    Run the following command from the src/whyhow_api/cli/ directory:

    $ cd src/whyhow_api/cli/
    $ python admin.py setup-collections --config-file collection_index_config.json
  4. Launch the WhyHow API Server

    main

    Start the API server using uvicorn. You can use the whyhow-locate utility to automatically resolve the correct path to the application.

    Standard launch:

    $ uvicorn src.whyhow_api.main:app

    Using the utility script:

    $ uvicorn $(whyhow-locate)

    Once running, you can access the Swagger UI documentation at http://localhost:8000/docs.

  5. Install WhyHow Knowledge Graph Studio

    main

    To install the Knowledge Graph Studio package, clone the repository and install via pip. This client requires Python 3.10 or higher.

    For standard installation:

    $ git clone git@github.com:whyhow-ai/knowledge-graph-studio.git
    $ cd knowledge-graph-studio
    $ pip install .

    For developers requiring an editable install with development and documentation dependencies:

    $ pip install -e .[dev,docs]
  6. Configure Environment Variables for WhyHow API

    main

    Before running the API, you must configure your environment variables. Copy the sample file to .env:

    $ cp .env.sample .env

    At a minimum, the following environment variables must be set:

    • WHYHOW__EMBEDDING__OPENAI__API_KEY: Your OpenAI API key for embeddings.
    • WHYHOW__GENERATIVE__OPENAI__API_KEY: Your OpenAI API key for generative tasks.
    • WHYHOW__MONGODB__USERNAME: Your MongoDB Atlas username.
    • WHYHOW__MONGODB__PASSWORD: Your MongoDB Atlas password.
    • WHYHOW__MONGODB__DATABASE_NAME: The name of your database (e.g., main).
    • WHYHOW__MONGODB__HOST: Your MongoDB Atlas host (e.g., xxx.xxx.mongodb.net).
  7. Run Knowledge Graph Studio with Docker

    main

    You can containerize the backend using Docker. Ensure you have completed the environment configuration and user creation steps first.

    Build the image:

    $ docker build --platform=linux/amd64 -t kg_engine:v1 .

    Run the image: Map the internal port 8000 to your desired host port using the OUTSIDE_PORT variable.

    $ OUTSIDE_PORT=1234
    $ docker run -it --rm -p $OUTSIDE_PORT:8000 kg_engine:v1
    $ docker build --platform=linux/amd64 -t kg_engine:v1 .
    $ OUTSIDE_PORT=1234
    $ docker run -it --rm -p $OUTSIDE_PORT:8000 kg_engine:v1
  8. Explore the WhyHow API Router structure

    main

    The WhyHow API is organized into several functional routers. When interacting with the API, you will use endpoints managed by these routers to perform operations on the following entities:

    • workspaces: Manage isolated environments.
    • schemas: Define data structures.
    • graphs: Manage knowledge graph structures.
    • triples: Handle subject-predicate-object relationships.
    • nodes: Manage individual entities in the graph.
    • documents: Manage source documents.
    • chunks: Manage text segments from documents.
    • users: Manage user accounts.
    • queries: Execute searches or graph traversals.
    • rules: Define logic or constraints.
    • tasks: Manage asynchronous or background operations.
  9. Manage document metadata and user-defined fields

    main

    Documents in WhyHow support two types of metadata:

    1. System Metadata (DocumentMetadata): Contains intrinsic file properties like size, format (e.g., pdf, csv, json, txt), and filename.
    2. User Metadata (user_metadata): A flexible dictionary allowing users to attach custom structured data. The schema for user_metadata follows a nested structure: dict[str, dict[str, AllowedUserMetadataTypes | list[AllowedUserMetadataTypes]]].

    You can update these fields using the DocumentUpdate model, which allows for partial updates to user_metadata and tags.

  10. Understand ObjectId handling and validation

    main

    The API uses MongoDB ObjectIds for entity identification. When interacting with the API, IDs are often represented as strings, but the underlying system validates them as ObjectId types.

    • AfterAnnotatedObjectId: A type that accepts either a str or an ObjectId. It automatically converts input to a string and then validates that the string is a valid BSON ObjectId.
    • Validation Error: If an invalid string is provided where an ID is expected, the API will raise a ValueError stating that the value is not a valid ObjectId.
  11. Understand Triple response formats

    main

    The API returns different representations of triples depending on the endpoint and visibility requirements:

    1. TripleOut: The full internal representation of a triple. Includes id (aliased from _id), head_node ID, tail_node ID, type, properties, graph ID, created_by ID, and associated chunks IDs.
    2. TripleWithId: A detailed response containing the full NodeWithId objects for both the head_node and tail_node, along with a RelationOut object (containing the relation name and properties).
    3. PublicTripleWithId: A version of the detailed response that provides NodeWithId objects for the head and tail but uses a simplified RelationOut structure, typically used for public-facing graph queries.
  12. Configure MongoDB connection settings

    main

    The MongoDB configuration is managed via SettingsMongoDB. To successfully generate a connection URI using the .uri property, you must provide username, password, and host.

    Required Fields:

    • username: The MongoDB username.
    • password: The MongoDB password (handled as a SecretStr).
    • host: The MongoDB host address.

    Other available fields:

    • database_name: The name of the database.
    • chunk_collection_name: The name of the collection for chunks (defaults to "chunk").
    • vector_search_embedding_size: The size of the embedding vectors (defaults to 1536).
    # Example environment variables for MongoDB
    WHYHOW__MONGODB__USERNAME=myuser
    WHYHOW__MONGODB__PASSWORD=mypassword
    WHYHOW__MONGODB__HOST=cluster0.mongodb.net
    WHYHOW__MONGODB__DATABASE_NAME=whyhow_db