Mnemosyne Documentation

Mnemosyne operates through a three-stage lifecycle to provide structured, local-first memory for agents:

1. Capture

After a Hermes turn completes, the provider automatically stores the interaction (user message, assistant response, tool calls, and execution context) in a local SQLite database. Memories are tagged with:

• Importance (0.0-1.0)
• Scope (session or global)
• Veracity (stated, inferred, tool, or imported)
• Metadata (optional, including expiration dates and named entities)

2. Recall

Recall is intentional rather than automatic. When an agent invokes mnemosyne_recall, the system performs a hybrid search combining:

• Vector similarity (semantic matches)
• FTS5 full-text search (keyword matches)
• Importance scoring (boosting significant memories)

Weights for these three components are tunable per-query to bias toward recency, relevance, or both.

3. Consolidate

To prevent the working memory from becoming overwhelming, the mnemosyne_sleep tool compresses old working memories into episodic summaries. This keeps the active context window small and ensures recall remains sharp during long-running sessions.

Mnemosyne's memory architecture is evolving into a layered system designed to extend the existing BEAM (working and episodic memory) engine. This model organizes information from raw events to high-level procedures, ensuring every synthesized memory maintains a link to its original evidence.

Memory Layers

Key Concepts

• Provenance: Every synthesized memory (L1-L4) must retain links to the raw supporting evidence (L0).
• Progressive Disclosure Recall: A recall strategy that returns the smallest useful information pack first (starting with L3/L4) and expands to L0 only when more evidence is required. This helps manage token budgets.

When the codebase version changes, generate a fresh codebase map to ensure documentation accurately reflects the current implementation. This involves cataloging all public interfaces to verify against documentation pages.

Key areas to catalog:

• Classes and Methods: e.g., BeamMemory in mnemosyne/core/beam.py, Mnemosyne in mnemosyne/core/memory.py.
• Tool Definitions: MCP tool definitions in mnemosyne/mcp_tools.py and Hermes plugin tools in plugin.yaml.
• Configuration: Environment variables (via os.getenv) and config.yaml keys.
• Hooks and Plugins: Hook registration in hermes_plugin/__init__.py and new import providers in mnemosyne/core/importers/.
• Dependencies: Entry points and dependencies in pyproject.toml.

Output the results to a structured JSON file (e.g., mnemosyne_codebase_surface.json) to facilitate comparison with documentation.

Mnemosyne uses a three-layer architecture to manage memory intake, storage, and retrieval, specifically designed to handle noise remediation through filtering and hygiene processes.

Layer 1: Pre-storage Filter (filters.py)

Before data is written to storage, it passes through a filter gate. This layer uses provider filters, core classifiers (regex + heuristics), and content sanitizers (binary extraction) to produce a WriteDecision. If the decision is reject, the content is dropped and logged; if allow, it proceeds to BEAM storage.

Layer 2: BEAM Storage (beam.py)

Data is stored in BEAM, which manages different memory types:

• working_mem: Hot memory with a 7-day TTL and 10K max capacity.
• episodic_mem: Summarized memories.
• memory_embeddings: The vector index.

BEAM also manages a Sleep cycle that triggers consolidate_to_episodic() to move data from working memory to episodic memory.

Layer 3: Post-storage Hygiene (hygiene.py)

This layer performs periodic audits of stored data. It uses audit_noise (scoring 0-1) and clean_noise (to delete, archive, or flag) to maintain database quality. A hygiene_audit_log maintains a full audit trail, and restore_archived() allows for reversibility.

Layer 4: Retrieval (polyphonic_recall.py + beam.py)

Retrieval is handled via Linear recall (FTS5 + working memory) and Polyphonic recall (hybrid vector + keyword). Ranking is determined by a combination of importance × veracity × Weibull recency × embedding sim.

Mnemosyne uses a sidecar table approach to handle media. Instead of creating a new specialized storage engine, media 'moments' (the text extracted from media) are stored as standard working_memory rows with a memory_type='artifact'. This allows moments to automatically inherit all existing features like vector search, full-text search (FTS), decay, and consolidation.

The Data Flow:

1. User Action: An explicit action is taken via mnemosyne media add, mnemosyne_media_register, or remember_media().
2. media_assets table: Stores the identity of the media (deterministic asset_id, modality, understanding_status, and archive_locator). Note: This table does not store raw bytes; it only holds references.
3. media_moments table: Stores the relationship between an asset and a specific point in time or space. It uses a span_kind (e.g., whole, time, page, char, box) and coordinates (t_start_ms, t_end_ms, etc.).
4. working_memory table: The actual text content of the moment is stored here as a standard memory row.
5. Recall: The system enriches results with media metadata (Phase 2), allows filtering by modality (Phase 3), and applies media-specific ranking weights (Phase 4).

   EXPLICIT USER ACTION ONLY  (no watcher, no polling, no directory scan)
   mnemosyne media add | mnemosyne_media_register | remember_media()
                          │
                          v
   ┌──────────────────────────────────────────────────────────────────┐
   │ media_assets            [NEW, core/media.py]                     │
   │ asset_id (deterministic) · ref_kind/ref_value · modality         │
   │ captured_at + precision · understanding_status · archive_locator │
   │ *** NO BLOB COLUMN. Bytes are never stored here. ***             │
   └──────────┬───────────────────────────────────────────────────────┘
              │ call_modality_describe()   [RFC 0002]
              │   -> DescribeResult{ moments: MomentDraft[] }  ... TEXT
              v
   ┌──────────────────────────────────────────────────────────────────┐
   │ media_moments           [NEW, core/media.py]                     │
   │ span_kind: whole | time | page | char | box                      │
   │ t_start_ms/t_end_ms · page_* · char_* · bbox · speaker           │
   │ UNIQUE(asset_id, kind, span_key) + INSERT OR IGNORE => idempotent│
   │ memory_id ──────────────┐  (soft ref, no FK per #503)            │
   └─────────────────────────┼────────────────────────────────────────┘
                             v
   ┌──────────────────────────────────────────────────────────────────┐
   │ working_memory          [EXISTING, unchanged schema]             │
   │ memory_type='artifact' · content = the moment's TEXT             │
   │ inherits: vec_working · fts_working · decay · sleep ·           │
   │           veracity · memory_events(sync) · mnemosyne reindex     │
   └─────────────────────────┬────────────────────────────────────────┘
                             v
   │ recall                                                            │
   │   phase 2: enrichment  -> result["media"] = {ref, span, locator} │
   │   phase 3: filter      -> EXISTS(...) correlated sub-select       │
   │   phase 4: _media_voice -> RRF k=60 (polyphonic_recall.py:734)   │