codemogger

Codemogger follows a five-step pipeline to make code searchable:

1. Scan: Walks the directory, respects .gitignore, and detects languages via file extensions.
2. Chunk: Uses tree-sitter (WASM) to parse files and extract top-level definitions (functions, structs, classes, impl blocks). Items exceeding 150 lines are split into sub-items.
3. Embed: Encodes chunks using a local embedding model (e.g., all-MiniLM-L6-v2).
4. Store: Saves chunks and embeddings into a single SQLite database using Turso, utilizing FTS (Full-Text Search) and vector extensions.
5. Search: Supports two modes:
   • Semantic: Uses vector cosine similarity for natural language queries.
   • Keyword: Uses FTS with weighted fields for precise identifier lookups.

Indexing is incremental; only files with changed SHA-256 hashes are re-processed on subsequent runs.

The CLI allows you to index a project and perform searches immediately.

1. Index a project: Scans the directory and creates a local database.
2. Search: Performs semantic or keyword searches over the indexed code.

Note: It is recommended to add the .codemogger/ directory to your .gitignore to avoid committing the local database.

# Index a project
codemogger index ./my-project

# Search
codemogger search "authentication middleware"

You can add codemogger to your coding agent's Model Context Protocol (MCP) configuration (e.g., Claude Code, OpenCode). This exposes three tools:

• codemogger_search: Semantic and keyword search over indexed code.
• codemogger_index: Index a codebase for the first time.
• codemogger_reindex: Update the index after modifying files.

Add the following to your MCP config:

{
  "mcpServers": {
    "codemogger": {
      "command": "npx",
      "args": ["-y", "codemogger", "mcp"]
    }
  }
}

You can install the codemogger CLI globally using npm, or run it on-demand using npx.

To install globally:

npm install -g codemogger

To run without installation, use npx followed by the command.

The CodeIndex class manages a persistent Store.

• Initialization: The store is lazily opened via getStore() when the first operation (index, search, etc.) is called.
• Verification: Before searching, the index performs a verifySearchable check to ensure the database file is not locked by another process and contains valid data (detecting issues like missing WAL files).
• Cleanup: Always call await index.close() when finished to properly close the database connection and release locks.

Codemogger can be run as a Model Context Protocol (MCP) server, exposing code indexing and semantic/keyword search as tools for AI agents. This allows an AI to navigate, understand, and search through a codebase using natural language or precise identifiers.

The server is typically started via the CLI mcp subcommand. Once running, it provides three primary tools: codemogger_search, codemogger_index, and codemogger_reindex.

Workflow:

1. Index: Use codemogger_index on a directory to perform the initial scan and embedding generation.
2. Search: Use codemogger_search to find relevant code chunks.
3. Maintain: Use codemogger_reindex after making significant changes to the codebase to keep the index fresh.

The following commands are available in the codemogger CLI:

• index <path>: Index a directory or project.
• search <query>: Search the indexed codebase.
• list: List all currently indexed codebases.

Example usage:

codemogger index ./my-project
codemogger search "authentication middleware"
codemogger list

# Index a directory
codemogger index ./my-project

# Search
codemogger search "authentication middleware"

# List indexed codebases
codemogger list

The search(query, opts?) method allows you to find relevant code chunks using three different modes:

• semantic: Uses vector search for natural language or conceptual queries.
• keyword: Uses Full-Text Search (FTS) for precise identifier or term lookups.
• hybrid: Combines both semantic and keyword results using Reciprocal Rank Fusion (RRF).

Use SearchOptions to control the limit, threshold (minimum score), and whether to includeSnippet in the results.

const results = await index.search('how is authentication handled?', {
  mode: 'hybrid',
  limit: 10,
  threshold: 0.5,
  includeSnippet: true
});

for (const res of results) {
  console.log(res.score, res.chunk.snippet);
}

The localEmbed function provides a way to generate vector embeddings for text locally using the all-MiniLM-L6-v2 model.

Key characteristics:

• Model: Uses all-MiniLM-L6-v2 (384 dimensions).
• Automatic Download: The model (~22MB) is downloaded on the first use and then cached locally.
• Memory Management: It processes texts in batches of 128 to balance throughput and prevent Out-Of-Memory (OOM) errors on typical hardware.
• Output: Returns an array of number arrays (number[][]), where each inner array represents the embedding vector for the corresponding input text.

import { localEmbed } from "./path/to/local";

const texts = ["Hello world", "Codemogger is great"];
const embeddings = await localEmbed(texts);
// embeddings is number[][]
console.log(embeddings[0].length); // 384

The formatJson function converts search query metadata and results into a structured, pretty-printed JSON string. This is useful for creating machine-readable outputs or API responses from search operations.

The resulting JSON object follows the JsonOutput interface structure:

import { formatJson } from './src/format/json.ts';

// Example usage:
const query = "example search";
const results = [{ /* ... SearchResult objects ... */ }];
const elapsedMs = 42;

const jsonString = formatJson(query, results, elapsedMs);
console.log(jsonString);