repository·main·Indexed 41 days ago
https://github.com/azurecosmosdb/cosmosaigraphAn implementation of the OmniRAG pattern that integrates Azure Cosmos DB's vector and hybrid search capabilities with Apache Jena's graph database. It provides a multi-source, intent-aware retrieval system for AI applications using a technology stack including Python, FastAPI, Azure OpenAI, semantic-kernel, and SPARQL 1.1.
CosmosAIGraph (caig) is a solution composed of two primary microservices:
These services are located in the web_app and graph_app directories of the repository and are packaged as Docker containers named caig_web and caig_graph respectively.
The Python implementation is located in the impl\app directory. Key subdirectories include:
src/models: Pydantic models for request/response payloads (webservice_models.py) and internal data structures (internal_models.py).src/services: Business logic implementation (e.g., ConfigService, AiService, DBService).src/util: Utility modules for filesystem access, prompt optimization, and SPARQL/OWL formatting.ontologies: OWL schema files (*.owl).rdf: RDF graph data files (*.nt).sparql: SPARQL query statements and Jinja2 templates.templates & views: Jinja2 templates for web and non-web use.tests: Unit tests using pytest.docker: Dockerfiles and docker-compose.yml files.A session reset via the brand link clears data in three layers:
localStorage data is wiped to remove client-side session state.conversation_id and other session variables.tmp/ directory associated with the conversation are removed:tmp/conv_{conversation_id}.jsontmp/ai_conv_{conversation_id}.jsonThe OmniRAG pattern is an advanced Retrieval Augmented Generation pattern designed to maximize result relevancy by detecting user intent and intelligently routing requests to multiple data sources.
Core tenets include:
By default, CosmosAIGraph uses an in-process, in-memory Jena model. You can switch to an external Apache Jena Fuseki triple store by setting the CAIG_GRAPH_BACKEND environment variable to fuseki.
AppGraphBuilder assembles the RDF model (ontology + triples from Cosmos DB/RDF/JSON).CAIG_GRAPH_BACKEND=fuseki is set, the graph_app connects to the Fuseki dataset at CAIG_FUSEKI_DATASET_URL and uploads the model via the SPARQL Graph Store Protocol./sparql_query, /sparql_update, and /add_documents are served by Fuseki via SPARQL 1.1./sparql_bom_query traversals, the graph_app fetches a single local snapshot of the Fuseki dataset and runs the traversal in-process.export CAIG_GRAPH_BACKEND=fusekiCosmosAIGraph does not require a pre-defined schema in Cosmos DB. The graph is constructed in-memory within the Apache Jena process by ingesting JSON documents from Cosmos DB during initialization.
To facilitate graph construction, you should include attributes in your JSON documents that represent relationships (edges). You can use a generic edges attribute containing a list of outgoing relationships, or use specific field names that the ingestion logic can map to RDF triples.
In the reference implementation for Python libraries, the following fields are used to build the graph:
developers: A list of developer contact identifiers.dependency_ids: A list of identifiers representing dependencies (e.g., pypi_<libname>).{
"developers": [
"contact@palletsprojects.com"
],
"dependency_ids": [
"pypi_asgiref",
"pypi_blinker",
"pypi_click",
"pypi_importlib_metadata",
"pypi_itsdangerous",
"pypi_jinja2",
"pypi_python_dotenv",
"pypi_werkzeug"
]
}The application uses ConfigService to manage configuration via environment variables. This approach is optimized for containerized deployments like Azure Container Apps (ACA) or Azure Kubernetes Service (AKS).
All application environment variables begin with the prefix CAIG_.
To use a .env file for local development, the project utilizes the python-dotenv library. Note: Do not commit .env files to source control as they may contain secrets.
Custom rules are dynamically injected into AI prompts to guide SPARQL generation. However, they only apply under specific conditions:
When you submit a natural language query in the NL2SPARQL console, any rules stored in sessionStorage under sparql_custom_rules are automatically loaded into a hidden field and sent to the backend. The resulting SPARQL will respect these domain-specific instructions.
Custom rules are applied in the OmniRAG Chat interface only when the "graph" RAG strategy is selected.
The solution is built using the following technologies:
CosmosAIGraph is composed of several key components that work together to provide AI-driven graph capabilities: