MemKraft

repository·main·Indexed 41 days ago

https://github.com/seojoonkim/memkraft

A local-first, Markdown-based compound knowledge system for AI agents. It implements an 'accountable memory loop' allowing agents to remember sourced facts, retrieve context within token budgets, and improve performance based on reported outcomes. Features include a Stable 3.x Lifecycle Core API, agentic search, prompt/skill evaluation, and support for Model Context Protocol (MCP) servers. Version 4.1.1.

Tokens
135.6K
Snippets
320
Records
545
Agent score
52%

What's inside MemKraft

  1. Overview of MemKraft 3.3.0 Execution Components

    main

    MemKraft 3.3.0 introduces the Execution State Kernel and the MemKraft Execution Protocol (MKEP/0). The core functionality is distributed across several key modules:

    • execution_state.py: Provides ExecutionStateMixin for validation, appending records, managing event sequences, and handling leases.
    • execution_projection.py: A pure function project(records, now) that uses a single _TRANSITIONS dictionary to derive state.
    • execution_handoff.py: Manages declaring, transitioning, exporting, and importing execution states, including a fail-closed redaction lint.
    • execution_protocol.py: Handles mkcjson encoding, digests, envelopes, error registries, and the closed dispatcher.
    • execution_cli.py: Provides CLI commands via memkraft exec call and memkraft exec state.
    • context_compiler.py: Implements dual opt-in for goal_id and execution_budget.
    • mcp.py: Provides 2 read-only tools with structured JSON encoding.
  2. Understand the MemKraft 3.0.1 Reliability Patch Goals

    main

    MemKraft 3.0.1 is a tightly-scoped, additive, and read-compatible reliability patch. Its primary objectives are:

    • Making lifecycle freshness observable.
    • Closing governance gaps for explicitly opted-in preview candidates.
    • Improving deterministic precision for English and Korean claim extraction.
    • Enhancing regression catching via the Memory Gym.

    Architecture Principles:

    • Markdown remains the durable user source material.
    • .memkraft/ logs are append-only.
    • Derived views and caches are rebuildable.
    • Truth is not auto-compiled on reads; instead, the system exposes the freshness state.
    • New candidate governance applies only to preview candidate sidecars and structured claims, not to legacy markdown search() calls.
  3. Understand the current MemKraft v2.1 benchmark stack

    main

    The current MemKraft benchmark implementation relies exclusively on longmemeval. This single-dependency setup uses a semantic majority vote approach and has reached a performance ceiling of 98.0% (N=50 oracle).

    Key components of the current stack include:

    • benchmarks/longmemeval/: The primary evaluation directory.
    • evaluator.py: Core evaluation logic.
    • harness.py: Test harness.
    • llm_judge.py: Uses a single LLM judge (currently Claude Haiku).
    • run_*.sh: Shell scripts for executing benchmarks.

    Note that the personamem/ directory exists but is currently unimplemented.

  4. Overview of the MemKraft V2.1 Core Refactor

    main

    The MemKraft V2.1 core refactor plan describes the decomposition of the monolithic core.py module into specialized mixin modules. This architectural change moves from a single large file to a modular mixin pattern managed via __init__.py.

    Key Architectural Changes:

    • Modular Mixins: Functionality is split into 9 new specialized modules (e.g., entity_ops.py, extract_nlp.py, agent_context.py).
    • Dependency Hierarchy: io_helpers.py serves as the foundation for all new modules. entity_ops.py is a central dependency for most other modules.
    • Public API Preservation: Despite the internal reorganization, the public API remains unchanged. Users can continue to use from memkraft import MemKraft and call methods like mk.track(), mk.search(), or mk.dream() as before.
  5. Migration Principles for MemKraft

    main

    MemKraft follows strict principles to ensure data integrity during version transitions:

    1. Three-Step Workflow: Every migration must follow the sequence: dry-run $\rightarrow$ apply $\rightarrow$ rollback. An apply without a dry-run or a rollback path is not permitted.
    2. User Markdown is Immutable: Migrations only affect the .memkraft/ sidecar and derived caches. User-provided markdown files (e.g., in memory/) are never rewritten or moved.
    3. Derived Caches are Disposable: Regenerable caches (like compiled_truth.jsonl) are not migrated; if a format changes, they are deleted and recreated.
    4. No Implicit Migrations: Disk formats are never changed silently by library imports or search calls. Changes only occur via memkraft migrate --apply.
    5. N-1 Read Compatibility: Readers are designed to be tolerant of one version prior (e.g., a 2.13 reader can read legacy records without envelopes). Writers always use the latest schema.
  6. When to use search_smart

    main

    The search_smart retriever is designed to provide lift in specific environments. It is most effective when the following conditions are met:

    1. High Noise/Large Corpus: The number of sessions is significantly larger than the number of relevant sessions (n_sessions >> n_relevant), requiring the retriever to actively filter noise.
    2. Identifiable Question Types: The query mix is dominated by identifiable types such as count or temporal questions.
    3. Parseable Dates: The corpus contains dates that can be parsed in YYYY-MM-DD format to leverage the search_temporal boost.

    Note: search_smart may not show significant improvements on 'Oracle' datasets where the number of retrieved files is already approximately equal to the number of relevant files.

  7. Construct an evidence chain for anomalous runs

    main

    When a completed run is identified as anomalous, you can anchor an inert sequence of evidence to it. Each link in the chain requires the exact predecessor type to maintain integrity.

    The Evidence Chain Sequence: retrospective → action → application → verification

    Verification Details: In delay_record_verification, the finding (caller-supplied opaque data) is independent of the verdict. The allowed verdict values are:

    • pass
    • fail
    • inconclusive
  8. Fallback behavior and constraints

    main

    The fallback mechanism is used when a task cannot be reliably executed via a recalled procedure.

    Constraints:

    • Single Call: The fallback(task) function is called exactly once per failed route.
    • Input Isolation: Only the original task is passed to the fallback. Recalled context (titles, lessons, trajectories) is strictly excluded to prevent injection.
    • Error Handling:
      • Fallback exceptions propagate after the single attempt.
      • If the fallback returns a non-string value, a TypeError is raised.
    • Accounting: A fallback route increments the model_calls count by 1.
  9. Understand the Project Memory Compiler contract and semantics

    main

    The Project Memory Compiler is designed around a strict semantic contract to ensure memory integrity and provenance. Key semantic concepts include:

    • Canonical IDs: Framed using SHA-256.
    • Temporal Semantics: Uses as_of timestamps and RFC3339 parsing.
    • Provenance: Every statement requires source or provenance. All source bytes are bound to receipts.
    • Conflict Handling: The system enforces zero silent conflict loss. Every source contradiction must produce either a retained conflict group or a visible aggregate omission diagnostic.
    • Authority & Privacy: Uses authority rank-only rules and privacy order/ceiling to manage access and visibility.
    • Determinism: The semantic digest is independent of input order, format, or the as_of time (permutation testing ensures input order does not change semantic output).
    • Safety: Enforces resource caps, path containment, and prevents symlink escapes via read-only adapters.
  10. Understand the MemKraft 4.0 Self-Evolving Agent Substrate contract

    main

    MemKraft 4.0 is designed as a substrate for self-evolving agents. The core design principle is that the system turns agent experience into evidence-backed, versioned, and reversible improvements, while strictly keeping execution and authority outside the memory substrate.

    To ensure safety and reliability, the 4.0 contract follows these constraints:

    • No direct execution: No MemKraft API can execute commands, deploy artifacts, schedule work, or grant permissions.
    • Authority separation: Model-generated fields cannot set authority_verified=true. All promotions require explicit authority.
    • Data safety: Secrets and artifact bodies are never stored in the ledger. Unauthorized profile/shared scope access fails before any append operation.
    • Deterministic projections: Every projection must be deterministic after JSONL line reordering.
    • Idempotency: All operations must be idempotent; mismatched retries must fail closed.
  11. Understand the MemKraft v2.1 Storage Layout

    main

    MemKraft v2.1 introduces a namespaced directory structure to support multi-agent shared memory and private workspaces. The storage is organized into a .memkraft/ metadata directory and an ns/ (namespace) directory.

    Directory Structure

    • .memkraft/:
      • agents/: JSON files for agent working memory.
      • channels/: JSON files for channel context.
      • tasks/: Task records and archives.
      • namespaces/: The core of v2.1, containing _registry.json (metadata and permissions), audit.jsonl (write audit logs), and locks/ (temporary lock files for concurrency control).
    • ns/:
      • [namespace_name]/ (e.g., zeon/, sion/, shared/):
        • entities/: Markdown files for entities (e.g., simon.md).
        • facts/: Fact records.
        • daily/: Daily logs.
        • channels/: (In shared namespaces) for collaborative context.
    • entities/, facts/, daily/: Legacy directories maintained for backward compatibility.
  12. Understand the MemKraft Execution Protocol (MKEP/0) Wire Contract

    main

    MKEP/0 is a runtime-neutral protocol for durable execution facts. It uses a specific request/response structure and a specialized JSON format called MKCJSON/1 to ensure deterministic digests.

    Request Structure

    Every request is a closed object containing:

    • mkep: Protocol identifier
    • kind: Type of request
    • request_id: Unique identifier for the request
    • op: The operation being performed
    • target: The target of the operation
    • args: Arguments for the operation
    • precondition: (Optional) Required for apply requests
    • now: (Optional) For clock-sensitive calls, callers must inject a timestamp in MKEP-TIME/1 format.

    Response Structure

    Responses are closed envelopes that indicate either success or error. They include a digest calculated over the fingerprint-normalized MKCJSON/1 representation of the object.

    MKCJSON/1 Specification

    To ensure deterministic hashing, MKCJSON/1 enforces the following:

    • Keys: Must be ASCII and match the regex ^[a-z][a-z0-9_]{0,63}$. Keys are sorted.
    • Types: Supports booleans, null, NFC strings, arrays, objects, and integers within the range $\pm(2^{53}-1)$.
    • Prohibited: Floats and lone surrogates are rejected.
    • Encoding: Compact separators, direct UTF-8 emission, and SHA-256 digests in lowercase hex over canonical logical objects (not the raw file bytes).