DOCSEARCHbyAlgolia
Submit Docs

memsearch

repository·main·Indexed 24 days ago

https://github.com/zilliztech/memsearch

A cross-platform semantic memory system for AI coding agents, version 0.4.16. It uses Markdown files as the source of truth and Milvus as a high-performance search index to enable persistent, searchable context. It features a Claude Code plugin that implements a three-layer progressive disclosure model (Search, Expand, and Transcript) for memory retrieval and supports multiple embedding providers, including a local-first ONNX bge-m3-onnx-int8 model and OpenAI.

Tokens
80.3K
Snippets
165
Records
371
Agent score
80%

What's inside memsearch

  1. Overview of memsearch

    main

    memsearch provides cross-platform semantic memory for AI coding agents. It allows memories to flow across different platforms like Claude Code, OpenClaw, OpenCode, and Codex CLI, ensuring that a conversation in one agent becomes searchable context in all others without extra setup.

    Key features include:

    • All Platforms, One Memory: Shared memory across multiple agent platforms.
    • Markdown as Source of Truth: Memories are stored as human-readable, editable, and version-controllable .md files. Milvus acts as a 'shadow index' (a derived, rebuildable cache).
    • Advanced Retrieval: Uses a 3-layer recall system (search → expand → transcript) with hybrid search (dense vector + BM25 sparse) and RRF reranking.
    • Efficient Syncing: Uses SHA-256 content hashing to skip unchanged content and a file watcher for real-time auto-indexing.

  2. Overview of the OpenClaw memsearch plugin

    main
    The memsearch plugin for OpenClaw is a TypeScript-based plugin (with kind: memory) that replaces the built-in memory-core with a hybrid semantic search system. It uses Milvus as a vector backend and stores memories as human-readable .md files, making them git-friendly and portable across different agent platforms like Claude Code, Codex, and OpenCode.

  3. Overview of memsearch CLI commands

    main

    The memsearch CLI allows you to index, search, and manage semantic memory for markdown knowledge bases.

    Core Commands:

    • index: Scan directories and index markdown files into the vector store.
    • search: Perform semantic search across indexed chunks using natural language.
    • watch: Monitor directories and automatically re-index when markdown files change.
    • compact: Compress stored memories into an LLM-generated summary.
    • reset: Drop all indexed data from the Milvus collection.
    • stats: Show statistics about the index (e.g., total chunk count).

    Progressive Disclosure Commands (Plugin Workflow):

    • expand: Show the full context/section around a specific memory chunk.
    • transcript: View original conversation turns from a JSONL transcript.
    $ memsearch --help
Usage: memsearch [OPTIONS] COMMAND [ARGS]...

  memsearch — semantic memory search for markdown knowledge bases.

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  compact     Compress stored memories into a summary.
  config      Manage memsearch configuration.
  expand      Expand a memory chunk to show full context.
  index       Index markdown files from PATHS.
  reset       Drop all indexed data.
  search      Search indexed memory for QUERY.
  stats       Show statistics about the index.
  transcript  View original conversation turns from a JSONL transcript.
  watch       Watch PATHS for markdown changes and auto-index.

  4. What is the Claude Code Plugin (memsearch)?

    main

    The memsearch plugin provides automatic, persistent memory for Claude Code. Unlike standard Claude sessions that start with a blank slate, this plugin automatically summarizes and indexes your conversations across sessions.

    Key benefits include:

    • Zero Intervention: No manual commands or saving required; capture and recall are fully automatic.
    • Semantic Recall: Claude autonomously searches past sessions when historical context is needed.
    • Transparent Storage: Memories are stored as plain .md files in a .memsearch/memory/ directory, making them human-readable, git-friendly, and portable.
    • Local Execution: Uses ONNX bge-m3 embeddings by default, running locally on your CPU with no API key required.
    • Efficient Context Management: Uses a "forked subagent" approach to perform memory searches in an isolated context, ensuring intermediate search results do not pollute your main conversation window.

  5. Overview of the Codex CLI Plugin

    main

    The Codex CLI Plugin provides semantic memory for the Codex CLI terminal coding agent. It uses shell hooks and a memory-recall skill to provide persistent memory capabilities that Codex lacks natively.

    Key characteristics include:

    • Hybrid Semantic Search: Uses progressive disclosure for memory retrieval.
    • Cross-platform Portability: Memories captured in Codex are searchable from other platforms like Claude Code, OpenClaw, or OpenCode.
    • Local Embeddings: Uses ONNX embeddings by default, meaning no OpenAI API key is required for the memory system itself (though Codex still uses OpenAI for the agent logic).
    • Architecture: Follows the same shell-hook-based architecture as the Claude Code plugin.

  6. Overview of the memsearch OpenCode Plugin

    main

    The memsearch OpenCode Plugin provides semantic memory for OpenCode. It is a TypeScript plugin that uses a background SQLite daemon to capture conversations and provides a three-layer memory recall system.

    Key advantages include:

    • Unified Memory Layer: Memories captured in OpenCode are accessible from other agents like Claude Code or Codex because they use a shared Milvus backend and human-readable .md storage format.
    • Hybrid Search: Combines semantic similarity (dense) with keyword matching (BM25) using Reciprocal Rank Fusion (RRF).
    • Progressive Disclosure: A three-layer recall mechanism consisting of search → expand → transcript.
    • Local-First: Uses ONNX bge-m3 embeddings by default, allowing it to run locally on a CPU without requiring API keys.

  7. Target audiences for memsearch

    main

    memsearch is designed for two distinct user groups:

    1. Agent Users: Users who want to install a plugin to gain persistent memory for their existing AI coding agents with zero manual commands or saving required.
    2. Agent Developers: Developers who want to build memory and harness engineering into their own custom agents using the provided CLI and Python API.

  8. Key features of memsearch

    main

    memsearch includes several advanced search and indexing capabilities:

    • Hybrid Search: Combines BM25 sparse search, dense vector search, and RRF (Reciprocal Rank Fusion) reranking for optimal recall.
    • Smart Dedup: Uses SHA-256 content hashing to skip indexing unchanged content during re-indexing.
    • Live Sync: Includes a file watcher that automatically indexes changes in real time.
    • Progressive Disclosure: Implements a 3-layer recall strategy: search $\rightarrow$ expand $\rightarrow$ transcript.
    • Pluggable Embeddings: Supports multiple embedding providers, including:
      • ONNX (runs locally and is free)
      • OpenAI
      • Google
      • Voyage
      • Jina
      • Mistral
      • Ollama

  9. Key features of memsearch

    main

    Core Capabilities

    • Automatic capture: Conversations are summarized and saved after each turn using the agent_end hook.
    • Three-layer progressive recall: Provides a workflow of search $\rightarrow$ expand $\rightarrow$ drill into original transcripts.
    • Cold-start context: Recent memories are automatically injected at agent startup via the before_agent_start hook.
    • Local Embeddings: Uses ONNX embedding by default, which runs locally on the CPU without requiring an API key.
    • Auditability: Allows tracing from a high-level summary back to the original JSONL transcript.

  10. Use cases for memsearch memory

    main

    You can use memsearch to enhance agent workflows in several ways:

    • Resume debugging threads: Ask how similar issues (e.g., Redis, Docker, database, or deployment) were resolved previously.
    • Recover decision rationale: Query why specific architectures, libraries, migration paths, or API designs were chosen.
    • Trace feature history: Understand feature evolution, including file changes and discussed tradeoffs.
    • Code archaeology: Ask when and why specific modules, configs, or workflows were modified.
    • Session recovery: Ask which previous conversation covered a specific topic to resume context.
    • Cross-agent context: Maintain a shared project memory across Claude Code, Codex CLI, OpenClaw, and OpenCode.

  11. How hybrid search works in memsearch

    main

    To ensure high-quality recall, memsearch employs a three-pronged Hybrid Search strategy:

    1. Dense vector search: Uses cosine similarity on embeddings to capture semantic meaning.
    2. BM25 sparse search: Uses keyword matching to catch exact terms like error codes, configuration values, or function names. The sparse vector is auto-generated by Milvus.
    3. RRF reranking: Uses Reciprocal Rank Fusion (with $k=60$) to merge the dense and sparse ranked lists into a single optimized result set.

  12. Understand the Three-Layer Progressive Recall model

    main

    The memsearch plugin uses a progressive disclosure strategy to provide increasing levels of detail, which helps manage context window usage and accuracy. The LLM autonomously decides which layer to use:

    LayerToolPurpose
    L1: Searchmemory_searchThe starting point. Finds top-K relevant snippets with relevance scores.
    L2: Expandmemory_getUsed when a snippet is relevant but needs more context. Returns the full markdown section and session anchors.
    L3: Transcriptmemory_transcriptUsed when the exact original exchange is needed (e.g., debugging specific failures). Returns the original JSONL dialogue.

    Workflow Example:

    1. User asks about a topic.
    2. LLM calls memory_search (L1).
    3. If a snippet looks relevant, LLM calls memory_get (L2) using the chunk_hash from L1.
    4. If the exact dialogue is still needed, LLM calls memory_transcript (L3).