GBrain: Postgres-native Personal Knowledge Brain

repository·master·Indexed 12 days ago

https://github.com/garrytan/gbrain

A synthesis and knowledge graph layer acting as persistent memory for AI agents. GBrain transforms raw data into synthesized answers with citations and automated entity-relationship mapping using a hybrid RAG search. It features a Deterministic Collector Pattern to separate data collection from LLM judgment, a system of 'recipes' for data integration, and BrainBench for measuring memory conformance across failure modes like know-to-ask and write-back.

Tokens
404.4K
Snippets
974
Records
1.6K
Agent score
98%

What's inside GBrain

  1. Overview of the GBrain Test Suite

    master

    The GBrain test suite is organized into several specialized domains to ensure the reliability of the brain engine, CLI, and data integrity. Key testing areas include:

    • Core Engine & Data: Tests for PGLite/Postgres parity, schema migrations, JSONB data safety (poisoning/UTF-16 surrogates), and file resolution.
    • CLI & Lifecycle: Tests for command structure, teardown contracts (ensuring clean exits and data flushing), and pipe completeness (ensuring piped stdout is not truncated).
    • Retrieval & Evaluation: Tests for search ranking (RRF, cosine similarity), intent classification, and the BrainBench memory-conformance suite.
    • Agentic Features: Tests for volunteer_context (push-based context), watch command (streaming loops), and resolver reachability.
    • Maintenance & Integrity: Tests for schema drift detection, skill conformance, and auto-fix capabilities (doctor --fix).
  2. What is GBrain?

    master

    GBrain is a synthesis and knowledge graph layer designed to serve as the 'brain' for AI agents. Unlike traditional search engines that return lists of raw pages, GBrain provides synthesized answers with citations and performs gap analysis to identify what information is missing from its memory.

    Key features include:

    • Synthesis Layer: Generates well-cited prose answers across people, companies, deals, and ideas, including explicit notes on knowledge gaps.
    • Self-Wiring Knowledge Graph: Automatically extracts entity references and creates typed edges (e.g., attended, works_at, invested_in, founded, advises) from page writes without requiring LLM calls.
    • Company Brain Support: Supports multi-user environments where data is scoped by login, ensuring users only access information they are authorized to see.
    • Autonomous Lifecycle: Designed to run as a daemon that ingests, enriches, and consolidates memory (the 'dream cycle') continuously.
  3. Explore GBrain tutorial roadmap

    master

    The following tutorials are currently in development:

    • VC Dealflow: Using gbrain whoknows, gbrain find-trajectory, and gbrain founder scorecard for managing founder and company data.
    • Vault Migration: Migrating Notion, Obsidian, or Roam vaults using gbrain schema detectsuggestreview-candidates.
    • Codebase Indexing: Initializing a brain in a code repo using voyage-code-3 embeddings and navigating via gbrain code-def, gbrain code-refs, and gbrain code-callers.
    • Fully Local GBrain: Running with Ollama or llama.cpp for privacy-first, air-gapped, or regulated environments.
    • The Dream Cycle: Setting up an overnight enrichment daemon for citations, deduplication, and automated scorecard generation.
  4. Evolve your brain's schema pack with schema-author

    master

    The schema-author skill is used to evolve your brain's schema pack by adding page types, link verbs, prefixes, and flags. This is an authoring-time task used to change the rules of your brain (e.g., "what types exist?"), rather than a write-time task (e.g., "where does this specific note go?").

    **Use this skill when you want to:

    • Add a new page type (e.g., researcher).
    • Propose new types based on corpus scans.
    • Backfill page.type on existing untyped pages.
    • Audit the health of your current schema pack.
    • Make a type 'expert-routable' or 'extractable'.

    **Do NOT use this skill for:

    • Filing a single specific page (use brain-taxonomist instead).
    • Checking schema as part of an EIIRP iteration (use eiirp instead).
    • Looking up a type's settings (use gbrain schema explain <type> instead).
    • Querying who knows about a topic (use gbrain whoknows <topic> instead).
  5. Design System Overview for GBrain Admin Dashboard

    master

    The GBrain Admin Dashboard is a dense, utilitarian developer infrastructure tool designed for managing OAuth agents, API keys, and monitoring requests. It follows an 'Industrial/Utilitarian' aesthetic, prioritizing function and data density over decoration.

    Core Principles:

    • Function-first: Every pixel must serve an informational purpose.
    • Zero Decoration: No gradients, shadows, or accent colors for decoration. The data is the interface.
    • Cockpit Mood: Designed for operators, not consumers. It is a high-density ops dashboard.
    • Dark Theme Only: No light mode is supported.
  6. Run real-world eval benchmarks against gbrain changes

    master

    GBrain provides an evaluation framework for maintainers and contributors to ensure changes to retrieval logic (search, ranking, embeddings, intent classification, query expansion, source boost, hybrid fusion) do not cause regressions.

    There are two primary evaluation paths:

    1. Retrieval Ranking (The Replay Loop): Uses captured real-world queries to compare current results against a baseline. This is used to detect if code changes shifted search results or increased latency.
    2. Memory Behavior (BrainBench): Uses gbrain eval brainbench to gate memory-related behaviors like Retrieval Reflex, conversation-to-facts write-back, and source isolation.

    For the NDJSON wire format used by these evaluations, refer to eval-capture.md.

  7. Understand the GBrain Test Suite Taxonomy

    master

    The GBrain repository uses a comprehensive test suite to ensure reliability across its various components. The test files are organized by functional area, covering everything from core data extraction and synchronization to complex OAuth 2.1 flows and MCP (Model Context Protocol) dispatching.

    Key functional areas covered by the test suite include:

    • Data Extraction & Processing: Tests for link extraction, timeline parsing, frontmatter parsing, and directory type inference (test/extract.test.ts, test/link-extraction.test.ts).
    • Synchronization & Concurrency: Extensive testing of sync logic, including concurrency thresholds, PGLite-specific serial execution, and error classification (test/sync.test.ts, test/sync-concurrency.test.ts, test/sync-failures.test.ts).
    • Minions & Job Management: Validation of the Minions job queue, including state machines, worker lifecycles, lock management, and dependency handling (test/minions.test.ts).
    • Security & Trust Boundaries: Tests for file upload security (symlink traversal, cwd confinement), query sanitization (prompt-injection stripping), and trust semantics (test/file-upload-security.test.ts, test/query-sanitization.test.ts, test/trust-boundary-contract.test.ts).
    • AI & LLM Integration: Testing of LLM-based features, including llms.txt generation, long-term memory evaluation (LongMemEval), and OpenAI-compatible multimodal embedding paths (test/build-llms.test.ts, test/eval-longmemeval.test.ts, test/openai-compat-multimodal.test.ts).
    • Infrastructure & Protocols: Coverage for OAuth 2.1 providers, MCP dispatching, and HTTP transport security/rate-limiting (test/oauth.test.ts, test/mcp-dispatch-summarize.test.ts, test/http-transport.test.ts).
  8. What is the Minion Orchestrator and when to use it

    master

    Minions is a Postgres-native job queue designed for durable, observable background work that must survive restarts, persist across sessions, or scale via parallel fan-out. It provides a single, steerable queue interface for two distinct types of work:

    1. Deterministic Shell Jobs: For reproducible command execution, ETL, or cron tasks where no LLM reasoning is required.
    2. LLM Subagent Jobs: For open-ended reasoning, research, or iterative agentic workflows.

    Routing Decision Guide:

    • Shell Job: Use when the user asks for a deterministic command/script or provides explicit cmd/argv parameters.
    • Subagent Job: Use when the user asks for research, reasoning, or an iterative agent.
    • Inline Execution: Consider for single, simple operations lasting <30s.
    • Minion Job: Use when the work requires restart durability or observability.
  9. What is GBrain?

    master

    GBrain is a compiled intelligence system designed for personal knowledge management at scale. Unlike a traditional note-taking app, it treats every page as an intelligence assessment consisting of two parts:

    1. Compiled Truth: A section representing the current best understanding of a topic, which is rewritten as new evidence emerges.
    2. Timeline: An append-only, immutable evidence trail of facts, references, or events.

    GBrain uses a Postgres-native backend (with pgvector) to provide high-performance hybrid search (combining vector semantic search and keyword search) and manages the intelligence lifecycle via AI agents.

  10. Overview of the four GBrain schema packs

    master

    There are four primary schema packs available, each serving a different functional purpose:

    • gbrain-base: The foundation upon which other packs extend.
    • gbrain-creator: Focuses on the atom and concept lifecycle (content creation).
    • gbrain-investor: Adds an investment lens (deals, theses, and bet resolutions).
    • gbrain-engineer: Acts as a bridge for technical learning, specifically for gstack integrations.
    • gbrain-everything: A meta-pack that combines the Creator, Investor, and Engineer lenses into a single active brain.
  11. What is a Schema Pack in GBrain

    master

    A Schema Pack is the single source of truth for the structure of your brain. It defines which directories exist, what types live in them, how the agent should infer types from file paths, and how different types are connected via link verbs.

    Every core operation—including filing, querying, routing experts, and fact extraction—consults the active schema pack at runtime. It is a dynamic artifact that allows the agent to understand the 'shape' of your data.

  12. Overview of GBrain Architecture

    master

    GBrain operates on two primary engines via a unified BrainEngine interface:

    • PGLite: A zero-config Postgres 17 via WASM engine, ideal for personal brains up to ~50K pages.
    • Postgres + pgvector: Used for shared, large, or multi-machine deployments (e.g., Supabase).

    Key Concepts:

    • Brain vs. Source: A brain is the database instance. A source is a specific repository (wiki, knowledge base, etc.) within that brain. Routing is determined by .gbrain-source dotfiles.
    • System of Record: Your knowledge is stored in a standard Git repository as markdown files. GBrain syncs these into Postgres. Deletions in Git result in soft-deletes in the database.