lat.md

repository·main·Indexed 23 days ago

https://github.com/1st1/lat.md

A system for maintaining a knowledge graph of a codebase using markdown files. It enables developers and AI agents to link architectural decisions and business logic to source code via wiki links, code links, and annotations. The tool includes a CLI for semantic search, reference validation, and integration with AI agents (Claude Code, Cursor, Copilot, Pi, OpenCode) via an MCP server and custom hooks.

Tokens
18.1K
Snippets
27
Records
125
Agent score
79%

What's inside lat.md

  1. Understand Strict vs Lenient link resolution

    main

    The behavior of link resolution depends on the command you are using:

    • Strict Context: Used by lat check and lat refs. Links must resolve unambiguously to a known section. Broken or ambiguous links result in errors.
    • Lenient Context: Used by lat locate and lat expand. These commands use tiered matching (exact → file stem → subsection tail → fuzzy) to allow for interactive exploration and approximate queries.
  2. How file walking and code reference scanning works

    main

    The project uses specific strategies for filesystem traversal and finding @lat: comments:

    File Walking

    All directory walking is centralized through src/walk.ts#walkEntries. This uses the ignore-walk package to ensure .gitignore rules are honored (filtering out .git/ and dotfiles). Results are not cached; every call re-walks the filesystem.

    Code Reference Scanning

    src/code-refs.ts#scanCodeRefs uses a two-tier strategy to find @lat: comments:

    1. Primary: Uses rg (ripgrep) for high-performance searching and file listing. It passes exclusions for lat.md/, .claude/, *.md, and sub-projects via --glob args.
    2. Fallback: A pure TypeScript implementation using walkFiles for discovery and filtering.

    Note: You can force the TypeScript fallback by setting the environment variable _LAT_DISABLE_RG=1.

  3. Understand the Section Preview output format

    main

    The Section Preview is a shared output format used by the locate, refs, and search commands. It provides a standardized way to view matches in the terminal. Each section in the preview is rendered as a bullet point (*) containing:

    1. Kind label: Indicates if the match is a File: (root section) or a Section: (subsection).
    2. Section ID: Displayed in [[wiki link]] syntax, where path segments are dimmed and the final segment is bold.
    3. Match reason: Shown in parentheses, such as (exact match), (section name match), or (fuzzy match, distance 2).
    4. Defined in: A label showing the file path (in cyan) and the line range.
    5. Body text: The first paragraph of the section, quoted with > (guaranteed to be ≤250 characters).

    Commands returning multiple results use formatResultList() to add a markdown ## heading and consistent spacing.

  4. Understand the lat.md package structure and publishing

    main

    The project is a pnpm workspace that publishes three distinct npm packages:

    1. lat.md (root): The main CLI tool. It includes dist/src and templates.
    2. @lat.md/embed: The embedding engine.
    3. @lat.md/embed-minilm-fp16: Bundled local weights.

    The root package depends on the two @lat.md/* packages via workspace:* protocols. During the pnpm publish process, these workspace references are automatically rewritten to the exact published versions to ensure stable pinning.

  5. Understand the Section data structure

    main

    The parser identifies 'sections' as a heading and all content following it until the next heading of equal or higher depth. Sections are organized into a hierarchical tree.

    Each section object contains the following properties:

    • id: A hierarchical path string (e.g., file#H1#H2). The first segment is the project-root-relative file path without the .md extension.
    • heading: The text of the heading.
    • depth: The markdown heading level (1–6).
    • file: Project-root-relative file path without .md (e.g., lat.md/dev-process).
    • filePath: Project-root-relative file path with extension (e.g., lat.md/dev-process.md).
    • children: An array of nested subsection objects.
    • startLine / endLine: The line numbers in the source file.
    • firstParagraph: The text of the first paragraph within the section.
  6. Use Wiki Links for internal and source code referencing

    main

    The lat.md format uses Obsidian-style wiki links [[target]] or [[target|alias]] to create connections between markdown sections and source code symbols.

    Targets are hierarchical paths starting from the vault root (the project directory). All paths must include the lat.md/ prefix.

    Resolution Rules:

    • [[foo]]: Links to the root section of foo.md.
    • [[foo#Bar]]: Links to heading Bar in foo.md. The path after # must be an exact heading chain (e.g., [[file#Heading1#Heading2]]).
    • [[path/foo#Bar]]: Fully qualified link to path/foo.md at heading Bar.

    You can link directly to symbols in supported languages (.ts, .tsx, .js, .jsx, .py, .rs, .go, .c, .h).

    • TypeScript/JavaScript: [[src/config.ts#getConfigDir]] (function) or [[src/server.ts#App#listen]] (method on class).
    • Python: [[file.py#my_func]] (functions, classes, methods, or module-level variables). Decorators are unwrapped automatically.
    • Rust: [[file.rs#Type#method]] (functions, structs, enums, traits, impl methods, consts, statics, type aliases).
    • Go: [[file.go#Type#Method]] (functions, types, methods with receivers, consts, vars).
    • C: [[file.h#Struct#field]] (functions, structs, fields, enums, typedefs, #define macros, variables).

    Note: Source code references always require the full path; short path disambiguation does not apply to source files.

    [[lat.md/guides/setup#Install]]
    [[src/config.ts#getConfigDir]]
    [[src/server.ts#App#listen]]
    [[src/lib.rs#Greeter#greet]]
  7. Understand lat.md syntax and referencing

    main

    The project uses a structured knowledge graph in lat.md/ using specific syntax for cross-referencing:

    Section IDs

    • Full form: lat.md/path/to/file#Heading#SubHeading (e.g., lat.md/tests/search#RAG Replay Tests).
    • Short form: Uses the bare filename if unique (e.g., search#RAG Replay Tests).

    Use [[target]] or [[target|alias]] to link between markdown sections. You can also link directly to source code:

    • TypeScript/JavaScript: [[src/foo.ts#myFunction]] or [[src/server.ts#App#listen]] for class methods.
    • Python: [[lib/utils.py#parse_args]].
    • Rust: [[src/lib.rs#Greeter#greet]].
    • Go: [[src/app.go#Greeter#Greet]].
    • C: [[src/app.h#Greeter]] for structs.

    Code References (Tying Code to Docs)

    To link source code to a specific documentation section, use a comment next to the relevant code block:

    • JS/TS/Rust/Go/C: // @lat: [[section-id]]
    • Python: # @lat: [[section-id]]
  8. Configure search backends and embedding models

    main

    The embedding backend used by lat search is determined by the index metadata. If no index exists, the backend is decided based on the presence of an API key.

    Key Resolution Priority:

    1. LAT_LLM_KEY (Environment Variable)
    2. LAT_LLM_KEY_FILE (Environment Variable)
    3. LAT_LLM_KEY_HELPER (Environment Variable)
    4. llm_key (Config file)

    Provider Detection based on Key Prefix:

    • No key: Uses local @lat.md/embed-minilm-fp16 (all-MiniLM-L6-v2, 384 dims).
    • sk-...: Uses OpenAI (text-embedding-3-small, 1536 dims).
    • vck_...: Uses Vercel AI Gateway (openai/text-embedding-3-small, 1536 dims).
    • sk-ant-...: Anthropic (not supported; will result in an error).

    Note: If you change your provider or key, you must run lat reindex to rebuild the index with the correct dimensions. The system will not silently switch models as vectors from different models are incompatible.

  9. Use Short Path Disambiguation for markdown files

    main

    For markdown files located inside the lat.md/ directory, you can use a shortened reference if the file stem is unique within the project.

    Example: If setup.md only exists under lat.md/guides/, you can use [[setup#Install]] instead of the full [[lat.md/guides/setup#Install]].

    Ambiguity Handling: If multiple files share the same stem (e.g., alpha/notes.md and beta/notes.md), the short form [[notes#Section]] is considered ambiguous. Running lat check --md will report an error listing the candidates and suggesting a fix.

  10. How the lat.md knowledge graph works

    main

    The lat.md system creates a knowledge graph for your codebase by using a lat.md/ directory at your project root. This graph connects high-level documentation to your actual source code using specific linking syntaxes.

    Linking Patterns

    • Wiki Links: Link between markdown sections using [[file#Section#Subsection]] syntax.
    • Code Links: Link markdown files directly to source code symbols using [[path/to/file.ext#symbol]] (e.g., [[src/auth.ts#validateToken]]).
    • Code Annotations: Tie implementation back to documentation by adding comments in your source files:
      • TypeScript/JavaScript: // @lat: [[section-id]]
      • Python: # @lat: [[section-id]]

    Example Structure

    my-project/
    ├── lat.md/
    │   ├── architecture.md    # system design, key decisions
    │   ├── auth.md            # authentication & authorization logic
    │   └── tests.md           # test specs (require-code-mention: true)
    ├── src/
    │   ├── auth.ts            # // @lat: [[auth#OAuth Flow]]
    │   └── server.ts          # // @lat: [[architecture#Request Pipeline]]
    └── ...
    my-project/
    ├── lat.md/
    │   ├── architecture.md    # system design, key decisions
    │   ├── auth.md            # authentication & authentication logic
    │   └── tests.md           # test specs (require-code-mention: true)
    ├── src/
    │   ├── auth.ts            # // @lat: [[auth#OAuth Flow]]
    │   └── server.ts          # // @lat: [[architecture#Request Pipeline]]
    └── ...
  11. How Wiki Links work in the parser

    main

    The parser implements custom Wiki Link syntax using a micromark and mdast extension. This allows for [[target]] style linking within markdown files.

    When a Wiki Link is parsed, it is represented as a wikiLink node in the mdast AST. This node contains:

    • value: The target string (the link destination).
    • data.alias: A string or null representing an optional alias.

    This implementation is specifically designed to be compatible with remark v11 and mdast-util-from-markdown v2.