AutoMem Documentation

The Neutral Agent Memory Benchmark (AMB) is used to evaluate AutoMem's performance across different memory scales and question types. The benchmark tracks two primary metrics:

1. Accuracy: The percentage of questions answered correctly.
2. Recall@5: The percentage of questions where the correct answer session was retrieved within the top 5 retrieved results.

Key benchmark suites include:

• LongMemEval full: 500 questions, testing various session types (knowledge-update, multi-session, etc.).
• LongMemEval mini: 30 stratified questions used as a canary.
• LoCoMo full: 10 conversations containing 1986 questions.

Note: For the most recent and authoritative performance claims, refer to the Neutral Agent Memory Benchmark (AMB) section in the documentation rather than the internal-harness engineering baselines.

The Remote MCP server exposes AutoMem over HTTPS, allowing cloud-based AI platforms (like ChatGPT, Claude.ai, and ElevenLabs) to access your memories without a local installation.

When to use it:

• Cloud AI Platforms: Required for ChatGPT, Claude.ai, Claude Mobile, and ElevenLabs Agents because they cannot connect to local MCP servers.
• Claude Desktop: Recommended if you want a shared memory connection that syncs across Claude Desktop, Claude.ai, and iOS.

When NOT to use it:

• Cursor IDE or Claude Code: You should use the local @verygoodplugins/mcp-automem package instead of the remote bridge.

When configuring AutoMem, choose your embedding dimension based on your deployment requirements:

Use 768d (text-embedding-3-small) if:

• You are cost-conscious.
• Storage is limited.
• Speed is more important than slight accuracy gains.
• You are using the system for personal or development purposes.
• Your dataset is small (<100k memories).

Use 3072d (text-embedding-3-large) if:

• You are in a production deployment.
• Accuracy is critical.
• You require complex multi-hop reasoning.
• You have a large dataset that benefits from higher precision.
• Storage and compute costs are acceptable for your use case.

AutoMem scripts are categorized by lifecycle tags, which indicate when and how they should be used. Understanding these tags is critical to avoid accidental data modification or running inappropriate tools in production:

• routine: Safe for normal, repeated operations (e.g., backups, monitoring).
• one-time: Idempotent scripts intended for single runs during upgrades or migrations. See docs/MIGRATIONS.md.
• recovery: "Break-glass" tools used only after data loss or corruption. See docs/MONITORING_AND_BACKUPS.md.
• dev: Helpers for local development or deployment.
• bench / lab: Evaluation and tuning harnesses for contributors; not required for standard AutoMem operation.

The LoCoMo-mini benchmark breaks down performance into several specific categories. Understanding these helps identify if regressions are specific to certain memory patterns:

• Single-hop: Basic retrieval.
• Temporal: Reasoning involving dates and time-based context.
• Multi-hop: Connecting multiple pieces of information.
• Open Domain: General knowledge retrieval.
• Complex (Category 5): High-level reasoning (requires an LLM judge).

Note: Temporal scores may appear artificially low if the evaluator is incorrectly comparing question dates to memory dates instead of answer dates. Complex scores may appear as 100% if the dataset lacks an answer field, causing the evaluator to match against an empty string.

AutoMem handles embedding dimension changes via the VECTOR_SIZE_AUTODETECT setting.

• Default behavior: VECTOR_SIZE_AUTODETECT=true automatically adopts your existing collection dimension on startup. This allows you to update the system without manual intervention for existing 3072d or 768d collections.
• Recommended setup: Use Voyage (voyage-4) at 1024d.
• OpenAI Fallback: If only an OpenAI key is provided, text-embedding-3-small is used and truncated to the configured VECTOR_SIZE via Matryoshka embeddings.

AutoMem uses two storage layers to provide hybrid recall:

1. FalkorDB (Graph Database): Acts as the canonical record. It stores memories as nodes connected by 11 authorable relationship types. If FalkorDB is unavailable, the API returns a 503 error.
2. Qdrant (Vector Database): Stores embeddings for every memory. It enables semantic search and hybrid queries (semantic similarity, graph traversal, temporal alignment, tag overlap, and importance).

If Qdrant is unavailable, the system can still serve recall in a degraded mode using only the graph. Recall is ranked by a 9-component score combining these layers.

Depending on your use case, use the following schedules for health checks and backups:

AutoMem performs background processes that modify the metadata object:

• Enrichment: Async and just-in-time enrichment processes parse existing metadata and merge generated entity data into metadata.entities. It also writes operational details under metadata.enrichment.
• Consolidation:
  • Creative consolidation adds graph relationships.
  • Identity consolidation deduplicates Entity nodes and can synthesize entity identity summaries.
  • Cluster consolidation may create MetaMemory nodes and SUMMARIZES relationships.

Note: Current consolidation does not merge two ordinary memory records into one. If future updates introduce memory merging, provenance fields like source_agents are intended to be merged rather than overwritten.