DOCSEARCHbyAlgolia
Submit Docs

Hindsight: Advanced Agent Memory System

repository·main·Indexed 28 days ago

https://github.com/vectorize-io/hindsight

An advanced agent memory system using biomimetic data structures to provide long-term memory and reasoning for AI agents. It serves as a pluggable memory layer for agent harnesses like OpenClaw and Claude Code, allowing agents to learn from prior interactions, preferences, and decisions. Includes comprehensive benchmarking suites for consolidation throughput, observation duplication, recall latency, and long-term memory evaluation (LongMemEval).

Tokens
628.5K
Snippets
1.3K
Records
2.8K
Agent score
94%

What's inside hindsight

  1. Overview of Multilingual Support

    main

    Hindsight automatically detects the language of your input and responds in the same language. Facts, entities, and reflect responses are preserved in their original language without translation to English. This behavior is driven by LLM prompt instructions rather than external detection libraries, allowing it to handle mixed-language content naturally.

    Key Behaviors

    • Retain: Extracts and stores facts in the original language.
    • Entities: Stored in their native script (e.g., 张伟 stays 张伟, not "Zhang Wei").
    • Reflect: Generates responses in the same language as the query.
    • Mixed Language: Gracefully handles content containing multiple languages (e.g., Chinese text with English company names).

  2. Overview of Hindsight Workflow and Automation Integrations

    main

    Hindsight can be integrated into workflow and automation platforms to add memory to automated processes:

    • n8n: Use the community node to drop retain/recall/reflect operations into any n8n workflow.
    • Zapier: Use the Zapier app for retain/recall/reflect actions and instant memory-event triggers.
    • Dify: Provides persistent memory for Dify apps.
    • Flowise: Provides memory nodes for Flowise flows.
    • Vapi: Provides persistent memory for Vapi voice agents.
    • Cloudflare OAuth Proxy: Allows running Hindsight behind Cloudflare Access via an OAuth proxy.

  3. Overview of Hindsight Agent Framework Integrations

    main

    Hindsight integrates with a wide range of agent frameworks to provide memory capabilities. Key integrations include:

    • LiteLLM: Provides proxy callbacks so every model proxied through LiteLLM gets memory with zero code changes.
    • CrewAI: Provides long-term memory tools for CrewAI agents.
    • Pydantic AI: Uses dependency-injected memory for Pydantic AI agents.
    • Vercel AI SDK / Vercel Chat: Provides persistent memory for Vercel AI SDK apps and drop-in memory for the Vercel AI Chatbot.
    • LangGraph / LangChain: Offers Memory Tools, Graph Nodes, and BaseStore adapter patterns.
    • LlamaIndex: Supports both agent-driven (BaseToolSpec) and automatic (BaseMemory) memory.
    • AutoGen / OpenAI Agents SDK: Uses FunctionTool-based memory.
    • Aider: The hindsight-aider wrapper recalls project memory before sessions and retains transcripts after.
    • OpenHands: Provides native MCP server config and recall/retain rules.
    • Smolagents: Provides retain/recall/reflect tools for HuggingFace Smolagents.

  5. Understand the Hindsight Event-Centric Graph

    main

    Unlike traditional knowledge graphs (like Neo4j) that map static relationships between entities, Hindsight uses an event-centric graph (a "scrapbook" model).

    • Structure: Data is organized around moments (conversations, observations, memories). Entities (people, places, concepts) are "stickers" attached to these memory pages rather than nodes connected by rigid arrows.
    • Handling Change: Because history is preserved in individual memory units, Hindsight handles changes (e.g., a user changing jobs) naturally. Instead of rewriting an edge, a new memory is created, preserving the old context.
    • Connection Signals: Hindsight finds connections between memories through three signals:
      1. Shared Entities: Two memories sharing the same entity sticker.
      2. Semantic Links: Precomputed proximity in embedding space.
      3. Causal Links: Explicitly extracted relationships like causes or prevents.

  6. Compare Codex Default vs. Hindsight Memory

    main

    Using Hindsight changes how Codex handles context and isolation:

    FeatureCodex DefaultWith Hindsight
    Memory across sessionsNoneAutomatic
    Memory setupManual AGENTS.mdExtracted from transcripts
    Recall mechanismFile content on startupSemantic search, injected per prompt
    Per-project isolationNoOptional via dynamicBankId
    Team shared memoryNoShared bank via Hindsight Cloud

  7. Use the @vectorize-io/hindsight-eve integration

    main

    The @vectorize-io/hindsight-eve package provides an integration helper to connect Eve agents to Hindsight using the Model Context Protocol (MCP).

    Key features include:

    • Auto-memory mode: Enables Eve to capture memories automatically without requiring explicit model tool-calling.
    • Assistant reply retention: As of version 0.2.1, the integration retains the assistant's replies in memory by default.

  9. Understand the architxt and Hindsight integration

    main

    architxt is a document processing pipeline that uses Hindsight as a durable, time-aware memory layer. It transforms fragmented enterprise documents (PDFs, Word, PowerPoint) into a queryable "Temporal Mosaic"—a composite view of the current state of entities (systems, services, capabilities) based on the latest available information across a corpus of dated documents.

    Key integration concepts:

    • Entities as Anchors: architxt detects and normalizes entities across documents. Using stable IDs (e.g., Billing Engine (SYS-001)) allows the system to map varied names (aliases) back to a single reference point, ensuring continuity even as naming conventions change.
    • Hindsight Retain/Observation: When architxt extracts facts, they are stored in Hindsight as timestamped, source-referenced observations. New documents do not overwrite old ones; they add newer facts, allowing the system to prefer the latest observations while maintaining historical context.
    • Hindsight Reflect/Mental Models: architxt uses reusable LLM prompts (Mental Models) to analyze entities from specific angles (e.g., interfaces, capabilities). These models can be refreshed as new documentation is ingested to keep summaries current.
    • Temporal Mosaic: Instead of a single "as-is" document, the system provides a composite view where the latest reliable knowledge for each entity is surfaced, regardless of which document it originated from.

  10. Understand Observations and Knowledge Consolidation

    main

    Hindsight automatically transforms raw facts into observations—deduplicated, evidence-grounded beliefs built from multiple memories. Unlike raw facts, observations represent synthesized knowledge (e.g., moving from "Alice prefers Python" to "Alice is a Python-focused developer who values readability").

    Key Features:

    • Deduplication: Consolidates overlapping facts into a single durable belief.
    • Grounding: Every observation is backed by specific source memories and exact quotes.
    • Evolution: Observations are refined (not just overwritten) when new evidence arrives, preserving the history of how a belief changed (e.g., a user switching from React to Vue).
    • Freshness Awareness: During reflect(), the system identifies "stale" observations that need verification against recent raw facts.
    • Retrieval: Observations are automatically included in recall() and reflect() operations.

  11. How Hindsight works with Cline lifecycle hooks

    main

    The Hindsight integration uses Cline's lifecycle hooks to manage memory deterministically. Because Cline does not provide a full transcript to hooks, the integration accumulates prompts locally and retains them at the end of a task.

    Cline hookHindsight Action
    TaskStartRecalls relevant context and injects it into the session.
    UserPromptSubmitRecalls memories for the specific message and records the prompt for later retention.
    TaskCompleteRetains the task's transcript and summary.
    TaskCancelRetains the partial transcript of the cancelled task.

    Recalled memories are injected into the prompt as a <hindsight_memories> context block.

  12. Understand TEMPR Multi-Strategy Retrieval

    main

    Hindsight uses TEMPR to run four search strategies in parallel to ensure high-quality retrieval. The results are combined using RRF (Reciprocal Rank Fusion) and refined via a Cross-Encoder.

    StrategyBest for
    SemanticConceptual similarity and paraphrasing
    Keyword (BM25)Names, technical terms, and exact matches
    GraphRelated entities and indirect connections
    TemporalTime-based queries like "last spring" or "in June"