DOCSEARCHbyAlgolia
Submit Docs

SDL-MCP (Symbol Delta Ledger)

repository·main·Indexed 19 days ago

https://github.com/glitterkill/sdl-mcp

A context budget layer for AI coding agents that indexes polyglot repositories into a symbol graph. It enables agents to navigate from compact metadata cards to specific source code windows via a controlled retrieval loop. The project includes a CLI, MCP tool interfaces (sdl.context, sdl.retrieve, sdl.workflow, sdl.file), and a plugin SDK for implementing custom language adapters using Tree-sitter or Regex.

Tokens
231.9K
Snippets
552
Records
997
Agent score
65%

What's inside sdl-mcp

  1. Understand the implemented acceptance paths in sdlbench

    main

    The sdlbench tool supports three distinct acceptance paths for running benchmarks and processing data:

    1. baseline: A fixture run performed without any product context tokens.
    2. sdl: A fixture run that includes SDL context-token accounting.
    3. import: A path used to parse transcripts into SessionRecord objects using the sdlbench import command.

    Note that crg and repomix are currently locked as dry-run competitors and are not used for standard fixture benchmarking until the path is stabilized.

  2. Use the file.write tool for token-efficient file updates

    main

    The file.write tool allows for surgical, token-efficient modifications to files instead of overwriting entire files. It supports six distinct write modes: full content overwrite, line range replacement, regex pattern replacement, JSON path updates, line insertion, and appending.

    By default, file.write creates a .bak file before modifying an existing file to ensure safety. You can disable this by setting createBackup to false.

    %%{init: {"theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#E7F8F2","primaryBorderColor":"#0F766E","primaryTextColor":"#102A43","secondaryColor":"#E8F1FF","secondaryBorderColor":"#2563EB","secondaryTextColor":"#102A43","tertiaryColor":"#FFF4D6","tertiaryBorderColor":"#B45309","tertiaryTextColor":"#102A43","lineColor":"#0F766E","textColor":"#102A43","fontFamily":"Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, Segoe UI, sans-serif"},"flowchart":{"curve":"basis","htmlLabels":true}}%% 
flowchart TD
    A[file.write request] e1@--> B{Write mode?}
    ...

  3. Technical Stack of SDL-MCP

    main

    SDL-MCP is built on the following technology stack:

    • Runtime: Node.js v24+ and TypeScript 5.9+ (strict, ESM).
    • Database: LadybugDB (an embedded graph database using the Kuzu engine, stored as a single file).
    • MCP SDK: @modelcontextprotocol/sdk ^1.29.0.
    • Transports: Supports stdio (for CLI agents) and HTTP/SSE (for network clients).
    • AST Parsing: tree-sitter 0.26.2 (via @keqingmoe/tree-sitter) using language grammars from sdl-mcp-tree-sitter-* wrapper packages.
    • Native Addon: Optional Rust implementation via napi-rs for multi-threaded Pass-1 indexing.
    • Embeddings: Local ONNX Runtime models:
      • jina-embeddings-v2-base-code for Symbols.
      • nomic-embed-text-v1.5 for FileSummary vectors.
    • Validation: All tool payloads and responses are validated using Zod schemas.

  4. Use the file.read tool for token-efficient reading

    main

    The file.read tool is designed for reading non-indexed files (such as configurations, documentation, templates, YAML, or JSON) within sdl.workflow steps. It is optimized to reduce token usage by offering targeted read modes instead of always reading entire files.

    Important: Do not use file.read for indexed source code files; use the dedicated SDL code tools for those instead.

  5. Understand the SDL-MCP Directory Structure

    main

    The SDL-MCP repository is organized into several functional domains. Key directories for developers include:

    • src/main.ts: The MCP server entry point using stdio transport.
    • src/server.ts: The MCPServer class responsible for tool dispatch and Zod validation.
    • src/indexer/: Contains the symbol extraction pipeline, including language adapters and Tree-sitter AST extraction.
    • src/delta/: Handles versioning, diffing, and blast radius calculations.
    • src/code/: Manages code windows, skeleton IR, and gating.
    • src/mcp/: Contains MCP-specific types, tools, errors, and telemetry.
    • src/agent/: The Autopilot orchestrator for planning and executing rungs.
    • native/: Rust addons implemented via napi-rs.
    • templates/: Contains MCP client configurations and agent instruction templates.

  6. What is SCIP and how does it integrate with SDL-MCP?

    main

    SCIP (Source Code Intelligence Protocol) is an open protocol used to emit compiler-grade code intelligence data. While SDL-MCP uses tree-sitter by default for fast, syntax-based symbol extraction, SCIP provides semantic depth that tree-sitter lacks.

    Key Differences

    • Tree-sitter (Default): Operates on syntax. It is fast and works across many languages but uses heuristics for cross-file call resolution and cannot see into external dependencies (like node_modules) or track interface implementations.
    • SCIP (Enriched): Operates on semantics. It uses actual compiler/type-checker data to provide exact symbol definitions, cross-file/cross-package references, relationship data (e.g., implements, overrides), and visibility into external dependencies.

    Integration Model

    Tree-sitter and SCIP are complementary. Tree-sitter provides the structural backbone (symbol extraction and skeleton IR), while SCIP upgrades heuristic edges to compiler-verified edges and adds relationship edges that syntax-only analysis misses. SCIP is treated as the highest-priority provider for provider-first indexing.

  7. Overview of the SDL-MCP Indexing Pipeline

    main

    The indexing process transforms repository files into a searchable graph structure through several stages:

    1. Pass 1: Local Extraction: Extracts symbols, imports, and calls from individual files.
    2. Pass 2: Cross-File Resolution: Uses a Call Resolver Registry to resolve edges between files, adding confidence scores to the graph.
    3. Finalization: Integrates embeddings and LLM-generated summaries into the LadybugDB graph.

    Additionally, Live Overlay Stores can merge real-time data from editor buffers with the static LadybugDB graph at read time.

  8. How hybrid retrieval works in SDL-MCP

    main

    SDL-MCP uses a hybrid retrieval pipeline that combines Full-Text Search (FTS), vector search, lexical search, and graph lanes. Instead of a mode selector, the system automatically combines available lanes and renormalizes weights.

    Results are fused using Reciprocal Rank Fusion (RRF) with a default rrfK = 60. RRF is used because it fuses rank positions rather than raw scores, making it robust against different score distributions between FTS and vector backends.

    If certain lanes are unavailable, the system performs Coverage-Aware Degradation:

    • hybrid: Both FTS and vector lanes are healthy.
    • hybrid-partial: Only one semantic lane is available.
    • lexical: Only lexical search is available.
    • graph-only: Structural seeds are used without a text lane.

  9. How the Context Ladder (Iris Gate) manages code access

    main

    The Context Ladder is an escalation mechanism that controls the amount of raw code an agent receives, moving from high-level summaries to full source code through four rungs:

    1. Rung 1: Symbol cards (~50-135 tokens per symbol) - Always available.
    2. Rung 2: Skeleton IR (~200 tokens per function) - Uses src/code/skeleton.ts to provide deterministic outlines (imports, type declarations, signatures) while eliding bodies.
    3. Rung 3: Hot-path excerpt (~500 tokens) - Uses src/code/hotpath.ts to find lines matching requested identifiers with configurable context lines.
    4. Rung 4: Full code window (Variable cost) - Provides the complete code block, gated by a "proof-of-need" policy.

    Proof-of-Need Gating Logic

    Requests for a full code window flow through two modules:

    1. policy/code-access.ts::decideCodeAccess: A pure module that applies a rule chain (caps, identifier requirements, budget, break-glass) to return approve, downgrade-to-skeleton, downgrade-to-hotpath, or deny.
    2. code/enforce.ts::enforceCodeWindow: Uses a WindowLoader (e.g., LadybugWindowLoader) to load text and verify that requested identifiers actually appear in the code.

  10. Configure Embedding Search models

    main

    The Semantic Engine uses hybrid retrieval (LadybugDB FTS + vector indexes). It supports two local ONNX models that are fetched via postinstall or lazily on first use:

    • jina-embeddings-v2-base-code (768-dim, ~110 MB): The default Symbol-lane model. It is optimized for code-to-code similarity and does not require natural-language summaries.
    • nomic-embed-text-v1.5 (768-dim, ~138 MB): The default FileSummary-lane model. It is optimized for prose and natural-language queries, benefiting significantly from LLM-generated summaries.

  11. Understand Cross-Platform CI Memory Sync Validation

    main

    The CI workflow ensures consistent behavior of the Symbol Delta Ledger (SDL) across Linux and Windows environments using three primary mechanisms:

    1. Parallel Execution: Running sync jobs simultaneously on both Linux (ubuntu-latest) and Windows (windows-latest).
    2. Artifact Comparison: Comparing generated artifacts to ensure consistency in metadata and data.
    3. Performance Validation: Ensuring both platforms operate within defined performance budgets.

    Consistency is verified by comparing attributes like commit_sha, branch, version_hash, artifact_hash, file_count, symbol_count, and edge_count.

  12. Configure scan scope and companion extensions

    main

    SDL-MCP filters out provider documents that fall outside the configured scan scope. To ensure important files are not accidentally excluded, the system uses "companion extensions" for specific languages:

    • C/C++: Includes .c, .h, .def, and .inc.
    • Python: Includes .pyi (stubs).

    For C/C++, the system tracks three distinct metrics to help you understand coverage:

    1. Scan scope: Every configured source file SDL-MCP scanned.
    2. SCIP semantic eligibility: The union of scan-scope files found in compile_commands.json plus provider-emitted C/C++ header/include documents within the scan scope.
    3. SCIP provider coverage: The count of provider documents that remain inside the scan scope after filtering unsafe or out-of-scope paths.