semantica

repository·main·Indexed 21 days ago

https://github.com/semantica-agi/semantica

A graph-native infrastructure layer for building accountable and explainable AI agent systems. Semantica provides deterministic reasoning, decision provenance (W3C PROV-O), and structured context management, including context graphs and explainable reasoning engines, to ensure AI decisions are traceable and outputs are auditable.

Tokens
457.5K
Snippets
1.2K
Records
1.5K
Agent score
80%

What's inside semantica

  1. Overview of the semantica.normalize module

    main

    The semantica.normalize module provides tools to standardize raw data before extraction and graph construction. This prevents data fragmentation (e.g., multiple nodes for the same company or different formats for the same date) by collapsing variants into canonical forms.

    Key capabilities include:

    • Text cleaning: Unicode normalization (NFC/NFKC), whitespace collapsing, and smart-quote/dash normalization.
    • Entity canonicalization: Resolving aliases and disambiguating entities using configurable maps.
    • Date normalization: Converting various formats (including relative dates) to ISO 8601.
    • Number conversion: Parsing strings like "$1.2B" into floats like 1200000000.0.
    • Language and Encoding: Detecting languages and repairing inconsistent byte encodings.
  2. Overview of the semantica.context module

    main

    semantica.context serves as the memory and decision layer for AI agents. It provides a unified interface for managing agentic state through several key capabilities:

    • Memory & Retrieval: Stores facts with provenance and supports embedding-backed semantic search.
    • Decision Tracking: Records decisions as first-class graph objects with full causal chains (tracing how decisions influence each other).
    • Graph-Backed Intelligence: Uses a knowledge graph for complex queries via multi-hop GraphRAG traversal.
    • Policy Enforcement: Manages versioned policies to ensure agent decisions comply with predefined rules.
    • Consistency: Allows agents to search their own history to prevent contradictory choices across different runs.
  3. Overview of Semantica Integrations

    main

    Semantica supports a wide range of integrations across different categories:

    Native Plugin Bundles

    Provides specialized skills and agents for editors including:

    • Claude Code
    • Cursor
    • Codex CLI
    • Windsurf
    • Cline
    • Continue
    • VS Code
    • OpenClaw

    Agentic Frameworks

    • Native Integration: Agno (pip install semantica[agno])
    • Via REST API & MCP: LangChain, LangGraph, CrewAI, LlamaIndex, AutoGen, OpenAI Agents SDK, and Google ADK.

    LLM Providers

    Supported via semantica.llms and LiteLLM, including OpenAI, Anthropic, Gemini, Mistral, Llama, Groq, Cohere, Azure, Bedrock, Ollama, DeepSeek, and HuggingFace.

  4. Overview of Semantica Capabilities

    main

    Semantica provides a graph-native infrastructure for building context-aware and accountable AI systems. Its capabilities are divided into four main pillars:

    1. Context & Decision Intelligence

    • Context Graphs: Structured, persistent graphs of entities and relationships using a temporal model (valid_from / valid_until) for point-in-time queries.
    • Decision Tracking: Lifecycle management for agent decisions via record_decision(), including causal chain analysis and similarity search to enforce consistency.

    2. Knowledge Engineering

    • Extraction: Named Entity Recognition (NER) and typed triplet extraction using pattern, ML, or LLM methods.
    • Ontology & Schema: Tools for visual editing, SHACL validation, and Datalog/SPARQL reasoning.
    • Deduplication: High-speed entity deduplication using blocking_v2, hybrid_v2, and semantic_v2 methods.

    3. Provenance & Auditability

    • Lineage Tracking: W3C PROV-O compliant lineage where every fact is linked to a source, featuring recorded_at timestamps and SHA-256 checksums for change management.
    • Compliance: Built-in support for audit-ready workflows required by HIPAA, SOX, GDPR, and FDA 21 CFR Part 11.

    4. Data Ingestion & Export

    • Ingestion: Supports documents (PDF, DOCX, HTML, etc.), structured data (JSON, CSV, Parquet, etc.), and various sources (SQL, Snowflake, MCP).
    • Storage: Integrates with major Vector Stores (FAISS, Pinecone, Weaviate, etc.) and Graph Stores (Neo4j, FalkorDB, etc.).
    • Export: Supports RDF (Turtle, JSON-LD), Tabular (Parquet, CSV), Graph (GraphML, GEXF), and Ontology (OWL, SKOS) formats.
  5. Overview of Semantica Modules

    main

    Semantica is organized into specialized modules that handle the lifecycle of knowledge graph construction and agentic reasoning. The pipeline typically follows this flow:

    1. Ingest: Loading data via FileIngestor, WebIngestor, ParquetIngestor, or XMLIngestor.
    2. Parse & Split: Converting raw files into structured text using DocumentParser or DoclingParser, then chunking text with TextSplitter.
    3. Normalize & Extract: Cleaning data with TextNormalizer and performing Named Entity Recognition (NER) or relation extraction using NERExtractor or TripletExtractor.
    4. Graph Construction: Building the knowledge graph using GraphBuilder and managing schemas with OntologyGenerator.
    5. Storage: Persisting data in VectorStore, GraphStore, or TripletStore.
    6. Reasoning & Context: Using Reasoner for logical inference and AgentContext to manage agent decisions and provenance.
    7. Orchestration: Managing the entire workflow using the Pipeline and PipelineBuilder modules.
  6. Bootstrap Knowledge Graphs with the Seed Module

    main

    The semantica.seed module allows you to provide a reliable, verified starting point for your knowledge graph by loading structured reference data such as taxonomies, product catalogs, or employee rosters. This process anchors entity extraction to known entities, reducing hallucinations and duplicate nodes.

    Key capabilities include:

    • Loading verified reference data (ISO codes, domain taxonomies, etc.).
    • Using SeedDataManager to merge extracted data onto foundation nodes without creating duplicates.
    • Supporting multiple formats: JSON, CSV, SQL (database), and API.
    • Providing deterministic test graph generation.
    • Implementing merge strategies (seed_first, extracted_first, merge) to handle conflicts between seed and extracted data.
  7. Use the TripletStore module for RDF storage

    main

    The semantica.triplet_store module provides W3C-standard RDF storage with SPARQL 1.1 query support. It offers a unified interface (TripletStore) that allows you to swap between different backends like Blazegraph, Apache Jena, and RDF4J by changing a single parameter. Use this module when you need semantic web compatibility, OWL-style reasoning, or SPARQL-based queries.

    from semantica.triplet_store import TripletStore
    from semantica.semantic_extract.types import Triplet
    
    store = TripletStore(
        backend="blazegraph",
        endpoint="http://localhost:9999/blazegraph/sparql"
    )
    
    t = Triplet(
        subject="http://example.org/apple_inc",
        predicate="http://example.org/founded_by",
        object="http://example.org/steve_jobs",
    )
    store.add_triplet(t)
  8. Use the Change Management Module for versioning

    main

    The semantica.change_management module provides version control, SHA-256 checksums, diff analysis, and rollback capabilities for knowledge graphs and ontologies. It supports compliance frameworks like HIPAA, SOX, GDPR, and FDA 21 CFR Part 11.

    Key Components

    • TemporalVersionManager: Manages snapshots, diffs, rollbacks, and per-node mutation history for Knowledge Graphs (KGs).
    • OntologyVersionManager: Manages schema versioning and structural diffs for OWL ontologies.
    • VersionStorage: Pluggable backends including InMemoryVersionStorage (for testing) and SQLiteVersionStorage (for production persistence).
    • Integrity Tools: compute_checksum() and verify_checksum() use SHA-256 to detect unauthorized modifications.
  9. What is GraphRAG and how does it work?

    main

    GraphRAG (Graph-Augmented Retrieval-Augmented Generation) enhances traditional RAG by combining vector similarity search with knowledge graph traversal. While traditional vector RAG finds documents similar to your query text, GraphRAG follows relationships between entities to find connected evidence across multiple documents, even if they don't mention your query terms directly.

    Key Benefits:

    • Multi-hop discovery: Finds facts that are several relationship steps away (e.g., Actor → Tool → Victim → Industry).
    • Connected evidence: Retrieves coherent chains of related entities rather than isolated fragments.
    • Explainability: Provides an audit trail showing exactly which entities and relationships led to the retrieved evidence.
  10. What is deduplication and entity merging in Semantica?

    main

    Deduplication is the process of identifying entities that refer to the same real-world object but appear as separate records (e.g., due to aliases, spelling variations, or different formatting) and merging them into a single canonical entity.

    Key Concepts:

    • Canonical entities: The single, authoritative representation of an object that all relationships in the knowledge graph point to.
    • Aliases: Alternative names or identifiers for the same entity (e.g., "APT29" and "Cozy Bear").
    • Entity resolution: The complete process including similarity calculation, duplicate detection, and merging.
    • Similarity algorithms: Semantica uses multiple signals including Jaro-Winkler (prefix-focused), Levenshtein (edit distance), exact match, cosine, property comparison, and vector embeddings.
    • Clustering: Using algorithms like Union-Find to group transitively related duplicates (if A matches B and B matches C, then A, B, and C are grouped together).
  11. What is a ContextGraph?

    main

    A ContextGraph is a thread-safe, in-memory property graph used to store entities as nodes and relationships as edges. It is designed for complex domains where entities require rich metadata and temporal context.

    Key features include:

    • Nodes: Represent entities (e.g., threat actors, vulnerabilities) with an ID, type, content, and metadata.
    • Edges: Represent relationships (e.g., 'uses', 'affects') with a type, weight, and metadata.
    • Metadata: Key-value pairs attached to both nodes and edges for domain-specific attributes.
    • Temporal Validity: Every node and edge can have a validity window, allowing for historical state tracking.
    • Built-in Capabilities: Includes Breadth-First Search (BFS) traversal and a FAISS vector index for semantic search.

    Note: ContextGraph is an in-memory data structure. To persist state for standalone graphs, use save_to_file(). If using AgentContext, use AgentContext.save() to save the graph, FAISS index, and memory simultaneously.

  12. Overview of Semantica Module Architecture

    main

    Semantica is organized into 27 independent modules across six logical layers. Each module is independently importable, allowing you to use only the specific components required for your task. The architecture is structured as follows:

    • Input Layer: Data ingestion and preparation (ingest, parse, split, normalize).
    • Core Processing: Intelligence and understanding (semantic_extract, kg, ontology, reasoning).
    • Storage: Persistent data storage (embeddings, vector_store, graph_store, triplet_store).
    • Quality Assurance: Data quality and consistency (deduplication, conflicts).
    • Context & Memory: Agent memory and decision tracking (context, provenance, change_management).
    • Output & Orchestration: Export, visualization, and workflows (export, visualization, pipeline, explorer).