DOCSEARCHbyAlgolia
Submit Docs

Engram Persistent Memory System

repository·main·Indexed 26 days ago

https://github.com/gentleman-programming/engram

A persistent memory system for AI coding agents providing a single-binary solution (Go + SQLite) to save, search, and retrieve context across sessions using the Model Context Protocol (MCP). It supports local SQLite storage, Git sync for portable memory chunks, and an optional Engram Cloud for shared project memory and replication. Engram integrates with various agents including Pi, Claude Code, Cursor, and VS Code Copilot, offering a suite of 20 MCP tools for memory lifecycle management.

Tokens
134.2K
Snippets
218
Records
797
Agent score
88%

What's inside Engram

  1. Overview of Engram for Pi Memory Paths

    main

    Engram connects Pi to persistent memory through two distinct paths depending on your workflow:

    1. Pi Extension (gentle-engram): Captures prompts and session events, injects the Memory Protocol, and provides compact, Pi-native mem_* tools via an HTTP server (ENGRAM_URL).
    2. MCP Tools (pi-mcp-adapter): Provides access to Engram's Model Context Protocol (MCP) surface for clients that use MCP directly. This uses a separate stdio path and requires an Engram binary (ENGRAM_BIN).

    Data Flow:

    • Pi events/tools $\rightarrow$ gentle-engram extension $\rightarrow$ ENGRAM_URL / engram serve $\rightarrow$ SQLite
    • Pi MCP tools $\rightarrow$ pi-mcp-adapter $\rightarrow$ ENGRAM_BIN / engram mcp $\rightarrow$ SQLite

  2. Overview of Cloud Autosync Specification

    main

    The Cloud Autosync Specification defines the behavioral contract for three core components:

    1. Background Autosync Manager: Handles the synchronization logic.
    2. Mutation REST Endpoints: Hosted on the cloud server to handle data changes.
    3. Status Adapter: Bridges autosync phases to the existing dashboard status pipeline.

    All requirements defined in this specification are designed to be testable via go test ./....

  3. Overview of Cloud User Token Management

    main
    Engram Cloud provides principal-based user and token management for human users. The system is designed to support server-side project access, token revocation, token hashing, and role-based administration. It is built to preserve existing sync routes, payload contracts, and legacy environment-token behavior during migration, ensuring that the cloud does not become the sole source of truth for local memory data.

  4. Overview of the obsidian-export specification

    main

    The obsidian-export is a CLI command designed to transform data from the Engram SQLite store into a structured, Obsidian-compatible markdown vault.

    Key features include:

    • Generation of markdown files with YAML frontmatter and wikilinks.
    • Creation of Hub Notes (Session Hubs and Topic Cluster Hubs).
    • Support for graph configuration bootstrapping.
    • An incremental sync mechanism to manage updates efficiently.
    • A built-in watch-mode daemon for automatic, periodic synchronization.

  5. Understand the Engram Mental Model

    main

    Engram provides coding agents with curated, searchable, and portable memory that persists after a session ends or a conversation is compacted.

    Key Principles:

    • Local-first: The SQLite database is the authoritative source of truth. Cloud functionality is an optional opt-in for replication and shared access.
    • Agent-driven: Engram is not a raw tool-call recorder; the agent decides which structured observations are worth saving.
    • Agent-agnostic: Integrations are achieved via Model Context Protocol (MCP), manual MCP configuration, or thin plugins.
    • Structured Memory: Agents use specific tools like mem_save, mem_session_summary, and mem_save_prompt to persist data.

  6. Understand the memory-conflict-surfacing mechanism

    main

    The memory-conflict-surfacing feature is designed to address semantic contradictions and invisible provenance in Engram's persistent memory. Instead of manual resolution, the agent automatically detects potential conflicts when saving new memories and manages the resolution process through structured judgments.

    Key capabilities:

    • Conflict Detection: When calling mem_save, the system identifies plausible candidates for conflict using FTS5 and BM25 scoring.
    • Automated Judgment: The agent uses mem_judge to record relations between memories (e.g., supersedes, conflicts_with) with full provenance.
    • Contextual Search: When calling mem_search, results include metadata like conflict_markers, supersedes, and superseded_by to inform the agent of the contested status of a memory.

  7. Understand Memory Conflict Surfacing Requirements

    main

    The memory-conflict-surfacing specification defines how Engram detects, marks, and resolves conflicting memory relations. Key behaviors include:

    • Conflict Detection: Triggered on save when similar titles or topic keys are detected.
    • Search Annotations: Search results are annotated with supersedes markers if a new relation replaces an old one, or contested markers if a relation is pending judgment.
    • Agent Judgment: Agents use the mem_judge tool to record verdicts on contested relations.
    • Multi-Actor Support: The schema supports multiple actors for the same relation pair, using unique sync_id values per row.
    • Provenance: Every relation row includes full provenance, including the actor (human or model) and the observation source.
    • Memory Decay: A decay schema is populated on save, assigning a review_after timestamp (defaulting to +6 months) to facilitate periodic reviews.
    • Local-only Relations: In Phase 1, relations may exist that are not present in sync_mutations.

  9. Choose an Engram storage mode

    main

    Engram supports three modes depending on your workflow requirements:

    • Local SQLite: Best for fast, private memory on a single machine.
    • Git sync: Best for portable, compressed memory chunks without requiring a hosted service.
    • Engram Cloud: Best for shared project memory, browser visibility, and replication across multiple machines or agents. Cloud is opt-in and project-scoped; Local SQLite remains the source of truth.

  10. Understand Cloud Dashboard Parity Gaps

    main

    The integrated engram repository currently provides only a status-only cloud dashboard, whereas the original engram-cloud repository provided a full server-rendered and htmx-enhanced UI.

    Current engram limitations:

    • Only a single dashboard.HandlerWithStatus(...) HTML page exists.
    • Token paste login at /dashboard/login is handled by cloudserver, not the dashboard package.
    • No dashboard static route mount (assets are placeholders).
    • Missing UI surfaces for browsers, projects, contributors, and admin.

    Required components for full parity:

    • Dashboard Package: Full route tree, rich templ components, helpers (pagination/formatting), middleware (cookie-auth context), and embedded static assets.
    • Cloudserver: Full route mounting via dashboard.Mount(...) and /dashboard/static/* support.
    • Auth: Bridging the gap between bearer-token validation and JWT user claims.
    • Cloudstore: Implementation of read-model API layers (e.g., ProjectStats*, Search*, Recent*) used by dashboard handlers.

  11. Understand the Local-First data model

    main

    Engram follows a local-first architecture. Even when cloud integration is enabled, your local SQLite database remains the single source of truth. Cloud capabilities are designed to extend, not replace, local behavior.

    Key constraints:

    • Unconfigured Cloud: If no cloud auth or endpoint is configured, all existing local commands (serve, mcp, search, context, local sync) behave exactly as they did previously.
    • Unenrolled Projects: Projects that have not been explicitly enrolled for cloud replication will not sync any data to the cloud.

  12. Cloud Autosync Synchronization Logic

    main

    Autosync provides silent background synchronization of local $\leftrightarrow$ cloud memories. It is designed with a 'local-first' invariant, meaning autosync must never block local writes if the cloud is unreachable.

    Key behaviors:

    • Project-Scoped Sync: The manager only pushes and pulls mutations for projects that are currently enrolled in the cloud service.
    • Lease-based Coordination: Uses AcquireSyncLease, MarkSyncHealthy, and MarkSyncFailed to coordinate sync activities.
    • Mutation Journaling: Uses a mutation journal with deduplication and tombstones to manage changes.
    • Conflict Resolution: Pull conflicts are handled via content-addressing deduplication to prevent data loss during concurrent writes.