CogDB Documentation

repository·master·Indexed 18 days ago

https://github.com/arun1729/cog

A persistent, embedded graph database for Python featuring Torque, a fluent traversal language. CogDB supports triple store models, JSON-based storage, and SIMD-optimized similarity search for word embeddings via SimSIMD. It includes capabilities for importing data from CSV, N-Triples, and Edgelist files, as well as an integrated HTTP server for remote graph access and mutation.

Tokens
3.9K
Snippets
20
Records
21
Agent score
62%

What's inside CogDB

  1. Use word embeddings for semantic search

    master

    CogDB supports word embeddings with SIMD-optimized similarity search using SimSIMD. You can load pre-trained embeddings, load from Gensim models, or add them manually to vertices. Once added, you can perform k-nearest neighbor searches or filter vertices based on similarity thresholds using the sim method.

    from cog.torque import Graph
    
    # Manual embedding addition
    g = Graph("fruits")
    g.put("orange", "type", "citrus")
    g.put_embedding("orange", [0.9, 0.8, 0.2, 0.1])
    
    # Find k-nearest neighbors
    # Search within graph vertices
    results = g.v().k_nearest("orange", k=2).all()
    
    # Or search ALL embeddings directly
    results = g.k_nearest("orange", k=2).all()
  2. Quick Start with CogDB

    master

    CogDB is a persistent, embedded graph database that lives inside your Python process. It uses a triple store model (Node → Edge → Node) and provides a fluent, chainable API called Torque for graph traversals.

    from cog.torque import Graph
    
    # Create a graph and add data
    g = Graph("social")
    g.put("alice", "follows", "bob")
    g.put("bob", "follows", "charlie")
    g.put("bob", "status", "active")
    
    # Query
    g.v("alice").out("follows").all()                    # → {'result': [{'id': 'bob'}]}
    g.v().has("status", "active").all()                  # → {'result': [{'id': 'bob'}]}
    g.v("alice").out("follows").out("follows").all()     # → {'result': [{'id': 'charlie'}]}
  3. Load data from CSV, Triples, or Edgelist files

    master

    CogDB supports importing data from several structured formats into a Graph instance.

    from cog.torque import Graph
    
    # Load from CSV (requires an ID column name)
    g = Graph("books")
    g.load_csv('test/test-data/books.csv', "book_id")
    
    # Load from N-Triples (RDF format)
    g = Graph(graph_name="people")
    g.load_triples("/path/to/triples.nt", "people")
    
    # Load from Edgelist
    g = Graph(graph_name="people")
    g.load_edgelist("/path/to/edgelist", "people")
  4. Serve a graph over HTTP

    master

    You can turn your local graph into an HTTP server to allow remote access.

    1. Server side: Call g.serve(port=8080).
    2. Client side: Use Graph.connect("http://<host>:<port>/<graph_name>") to connect to the remote instance and query it using the same Torque API.
    # Server
    from cog.torque import Graph
    g = Graph("social")
    g.put("alice", "follows", "bob")
    g.serve(port=8080)
    
    # Client
    remote = Graph.connect("http://localhost:8080/social")
    print(remote.v("alice").out("follows").all())
  5. Install CogDB via pip

    master

    Install the cogdb package using pip to get started with the embedded graph database.

    pip install cogdb
  6. Optimize performance with `flush_interval`

    master

    Control how often data is written to disk to balance safety and speed during bulk inserts.

    flush_intervalBehaviorUse Case
    1 (default)Flush every writeInteractive, safe
    > 1Async flush every N writesBulk inserts
    0Manual only (sync())Maximum speed

    If using flush_interval=0, you must call g.sync() to ensure data is persisted.

    # Fast mode: flush every 100 writes
    g = Graph("mydb", flush_interval=100)
    
    # Maximum speed: manual flush only
    g = Graph("mydb", flush_interval=0)
    g.put_batch(large_dataset)
    g.sync()
  7. Configure Cog settings

    master

    If no configuration is provided, Cog uses default values for COG_PATH_PREFIX (/tmp) and COG_HOME (cog-test). You can override these by importing the config module.

    from cog import config
    
    config.COG_HOME = "app1_home"
    # Use the config object when initializing Cog
    cog = Cog(config)
  8. How CogDB HTTP Server routing works

    master

    The CogDB HTTP Server supports multiple graphs on a single port using path-based routing. The URL structure determines the target graph and the action to perform.

    URL Patterns:

    • /{graph_name}: Accesses the status page for the specified graph.
    • /{graph_name}/status: Accesses the status page for the specified graph.
    • /{graph_name}/stats: Returns JSON statistics for the specified graph.
    • /{graph_name}/query: Executes a Torque query (POST).
    • /{graph_name}/mutate: Performs write operations (POST).
    • /: Accesses the index page listing all registered graphs.
  9. Load pre-trained embeddings (GloVe or Gensim)

    master

    You can quickly populate your graph with embeddings using GloVe files or Gensim models.

    # Load GloVe embeddings
    count = g.load_glove("glove.6B.100d.txt", limit=50000)
    
    # Load from Gensim model
    from gensim.models import Word2Vec
    model = Word2Vec(sentences)
    count = g.load_gensim(model)
  10. Delete edges with `drop`

    master

    Remove a specific relationship (edge) between two nodes using the drop method.

    g.drop("bob", "follows", "fred")
  11. Querying with Torque: Basic Traversals

    master

    Torque provides a fluent API for navigating the graph:

    • v(id): Start traversal from a specific vertex.
    • v(): Scan all vertices.
    • out(label): Follow outgoing edges with the specified label.
    • in(label): Follow incoming edges with the specified label.
    • both(label): Follow edges in both directions.
    • inc(): Include the edges in the result set.
    • all(): Execute the traversal and return all results.
    # Follow outgoing edges
    g.v("bob").out().all()
    
    # Filter vertices by property
    g.v().has("status", "active").all()
    
    # Include edges in results
    g.v().has("follows", "fred").inc().all('e')
    
    # Bidirectional traversal
    g.v("bob").both("follows").all()
  12. Filter vertices by similarity

    master

    The sim method allows you to filter vertices based on their cosine similarity to a target vector or vertex. You can use comparison operators like > or range operators like in.

    # Filter by similarity threshold (greater than 0.9)
    g.v().sim('orange', '>', 0.9).all()
    
    # Find items in a similarity range
    g.v().sim('orange', 'in', [0.5, 0.8]).all()
    
    # Combine graph traversal with similarity
    # Find citrus fruits similar to orange
    g.v().has("type", "citrus").sim("orange", ">", 0.8).all()