Claude Self-Reflect

The Saga Eval Kit is a Phase-2 experiment harness containing scripts, prompts, specs, and metadata used to reproduce the experiments from the saga/reinstatement paper. It is designed to allow operators to re-run the research methodology on their own private corpora.

The kit is organized into three distinct experimental modules:

1. E2 (Ratification-derived graded gold): Focuses on building graded relevance gold using operator dialog-acts and external ship ledgers.
2. E1 (Seven-arm ablation): A retrieval harness testing seven different retrieval arms (from kNN to full walk) against a shared index.
3. E3 (Controlled contamination): A dose-response injection experiment testing baseline, transcript injection, sham transcripts, and scripted re-ask cycles.

Note: The kit contains only metadata (IDs, grades, hashes, protocol). Private data such as conversation digests, raw extraction outputs, and corpus databases are not included for privacy reasons.

Overview

claude-mem is a persistent memory compression system designed for Claude Code. It aims to solve the 'cold start' problem by automatically capturing session activity, compressing tool observations using the Claude Agent SDK, and injecting relevant context into future sessions.

Key Features

• Automatic Session Capture: Hooks into Claude Code lifecycle events.
• AI Summarization: Generates semantic summaries of tool observations.
• Context Injection: Automatically provides relevant memory at the start of new sessions.
• Skill-based Search: Uses a mem-search skill for retrieval.
• Progressive Disclosure: Returns compact results first, followed by timeline context and full observations.
• Privacy Control: Supports <private>...</private> tags to prevent specific content from being stored.
• Local Web Viewer: Provides a React-based UI on a local worker port.

Architecture

The system relies on a multi-component local stack:

• Runtime/Engines: Node, Bun, and uv (for Python vector-search dependencies).
• Databases: SQLite (~/.claude-mem/claude-mem.db) and Chroma vector database.
• Services: A Bun-managed worker service exposing an Express/HTTP API (typically on port 37777).
• Integration: TypeScript hooks, MCP search tooling, and Claude Agent SDK.

Claude Self-Reflect (CSR) is a local-first memory system designed to solve the 'forgetting problem' in AI agents like Claude. It provides a single 44MB binary that gives Claude persistent memory of past sessions, architectures, and bug fixes without requiring external databases, containers, or API keys.

Key features include:

• Local Execution: Everything runs in a single process using SQLite for storage and HNSW for sub-millisecond vector search.
• Automatic Context: Once installed, past context appears automatically in Claude's context window when relevant.
• Code-Awareness: Uses AST (Abstract Syntax Tree) to perform search across 6 languages.
• Multi-Source Memory: (v9.4+) Remembers transcripts, task outcomes, plan documents, and a cross-project session registry.
• MCP Integration: Provides 15 MCP tools for explicit search, such as csr_code_graph to link code symbols to their conversational history.

The T3 (Cross-Source Multi-Hop) Benchmark evaluates the effectiveness of the 'reinstatement walk' (R) against 'kNN + FTS RRF' (K) retrieval mechanisms.

Key Findings:

• Gate M (Mechanical Set): The reinstatement walk (R) failed to outperform flat retrieval (K). K achieved a recall@5 of 0.783 compared to R's 0.545. The mechanism for this win is attributed to the vector channel doing the heavy lifting, while the reinstatement walk's defenses actually dilute exact-token evidence needed for receipt lookups.
• Gate D (Deep Set): No significant difference was found between R and K on multi-hop subsets, with results mildly favoring K.
• Memory Importance: The 'no-memory' arm scored 0/27, proving that retrieved context is essential for correct answers in this corpus.

Core Conclusion: The 'reinstatement walk' does not provide a cost-effective advantage over one-shot kNN+FTS for multi-hop provenance questions on this corpus. Instead, the project's value lies in the receipts layer, the resolution ledger, and provenance-aware reranking.

The benchmark results indicate that while memory is critical for preventing re-litigation, the specific method of retrieval (Reinstatement vs. kNN) matters differently depending on the complexity of the task.

Key Findings

• Memory vs. No-Memory: Memory arms (R and K) significantly outperform the no-memory arm on settled-for-cause tasks, nearly doubling reason-quality scores (10/12 and 9/12 vs 6/12). Memory prevents the dangerous failure mode of accepting requests that re-break previously shipped fixes.
• Reinstatement vs. kNN (Single-hop): On single-hop tasks (where the justification is found in one step), Reinstatement Recall (R) and kNN (K) perform identically in both action accuracy and reason quality. The advantage of Reinstatement Recall is localized to multi-hop provenance queries.
• Conservatism Bias: All arms (including no-memory) exhibit a uniform conservatism bias on should-change tasks, often defaulting to NEEDS-INFO instead of accepting valid changes.
• No Negative Over-generalization: Memory does not cause the agent to incorrectly apply past failures to structurally different follow-up tasks (irrelevant-past tasks scored 6/6 across all arms).

The landing page uses a fixed 12-column CSS grid on desktop with specific row spans for an eight-card hero bento layout.

Desktop Grid Configuration:

• Columns: 12 columns (repeat(12, minmax(0, 1fr))).
• Rows: Three fixed rows (300px, 250px, 210px).
• Card Mapping:
  • .card--forgetting: Col 1 / Span 5, Row 1
  • .card--active: Col 6 / Span 4, Row 1
  • .card--search: Col 10 / Span 3, Row 1
  • .card--import: Col 1 / Span 4, Row 2
  • .card--binary: Col 5 / Span 3, Row 2
  • .card--pipeline: Col 8 / Span 5, Row 2
  • .card--privacy: Col 1 / Span 3, Row 3
  • .card--install: Col 4 / Span 4, Row 3

Responsive Breakpoints:

• 768px: Switches to an 8-column grid. .card--forgetting and .card--pipeline span the full width. Other cards span 4 columns.
• 480px: Switches to a flex column layout. Bento cards get min-height: auto and specific padding/blur settings.

Note: If a ninth documentation entry card is required, place it at .docs-entry-card { grid-column: 8 / span 5; grid-row: 3; } below the primary viewport.

.bento-grid {
  display: grid;
  grid-template-columns: repeat(12, minmax(0, 1fr));
  grid-template-rows: 300px 250px 210px;
  gap: 16px;
  align-items: stretch;
}

.card--forgetting { grid-column: 1 / span 5; grid-row: 1; }
.card--active { grid-column: 6 / span 4; grid-row: 1; }
.card--search { grid-column: 10 / span 3; grid-row: 1; }
.card--import { grid-column: 1 / span 4; grid-row: 2; }
.card--binary { grid-column: 5 / span 3; grid-row: 2; }
.card--pipeline { grid-column: 8 / span 5; grid-row: 2; }
.card--privacy { grid-column: 1 / span 3; grid-row: 3; }
.card--install { grid-column: 4 / span 4; grid-row: 3; }