DOCSEARCHbyAlgolia
Submit Docs

NextPlaid & ColGREP

repository·main·Indexed 19 days ago

https://github.com/lightonai/next-plaid

NextPlaid is a local-first, multi-vector search engine for high-performance retrieval. It powers ColGREP, a semantic code search tool that combines ColBERT semantic ranking with regex filtering and FTS5 trigram keyword search. The ecosystem includes the colgrep-parser Python SDK for extracting structural code units (functions, classes, etc.) from a wide range of programming languages and configuration formats using tree-sitter AST analysis.

Tokens
73.1K
Snippets
247
Records
316
Agent score
64%

What's inside NextPlaid

  1. Important colgrep rules and defaults

    main

    When using colgrep, keep the following behaviors and rules in mind:

    Default Exclusions colgrep automatically excludes the following directories to reduce noise:

    • .git
    • node_modules
    • target
    • .venv
    • __pycache__

    Search Scope

    • If you run colgrep from a subdirectory, results are restricted to that subdirectory. To search the entire project, specify . or .. as the path.

    Pattern Logic

    • -F (Fixed string) takes precedence over -E (Extended regex), similar to standard grep behavior.
    • Multiple --include patterns use OR logic (a file is included if it matches any of the patterns).
    • Brace expansion (e.g., *.{rs,md,py}) is supported for matching multiple file types.

    Best Practices

    • Use colgrep as your primary search tool instead of standard Search, Grep, or Glob.
    • Increase --results (or -k) to 20-30 when exploring or trying to understand a system.
    • Use -e for hybrid text+semantic filtering to narrow down semantic results with known patterns.

  2. Optimize indexing speed and memory with --parallel

    main

    The --parallel setting controls the number of ONNX encoding sessions. This is a persistent setting that affects all future indexing.

    • CPU: Default is the CPU core count (max 16). Increasing this speeds up indexing linearly with minimal memory impact.
    • GPU / CoreML / DirectML: Default is 1. Because each session duplicates the model in device memory, you should generally keep this at 1 unless you have significant device memory available.

    Use --parallel 0 to reset to the automatic default.

    # Use more sessions for faster indexing on multi-core machines
colgrep settings --parallel 8

# Use a single session to minimise memory (recommended on GPU/CoreML)
colgrep settings --parallel 1

# Reset to the automatic default
colgrep settings --parallel 0

  3. Understand NextPlaid update modes

    main

    NextPlaid uses three incremental update strategies based on the number of documents and the current index state:

    ModeConditionBehavior
    Rebuildnum_docs <= start_from_scratch (default: 999)Load existing embeddings + new, full K-means rebuild
    Buffernew_docs < buffer_size (default: 100)Assign to existing centroids, buffer for later
    Expandnew_docs >= buffer_sizeFind outlier embeddings, expand centroids via K-means, re-index buffer + new

  4. Configure Hybrid Search modes

    main

    By default, colgrep uses Hybrid Search, fusing ColBERT semantic search with FTS5 trigram keyword search via Reciprocal Rank Fusion (RRF). This improves recall for exact identifier matches.

    You can toggle this behavior persistently using colgrep settings, or use the --semantic-only CLI flag as a one-shot override for a single query.

    # Disable hybrid search (pure semantic mode) persistently
colgrep settings --no-hybrid-search

# Re-enable hybrid search (default) persistently
colgrep settings --hybrid-search

# One-shot override (does not change settings)
colgrep "query" --semantic-only

  5. Configure Hardware Acceleration and Execution Providers

    main

    The ExecutionProvider enum controls which backend ONNX Runtime uses.

    • ExecutionProvider::Auto: Tries providers in order: CUDA → TensorRT → CoreML → DirectML → CPU.
    • ExecutionProvider::Cpu: Forces CPU only.
    • ExecutionProvider::Cuda: Requires cuda feature.
    • ExecutionProvider::TensorRT: Requires tensorrt feature.
    • ExecutionProvider::CoreML: Requires coreml feature.
    • ExecutionProvider::DirectML: Requires directml feature.

    To bypass all GPU providers and force CPU, set the environment variable NEXT_PLAID_FORCE_CPU=1.

    // Example of forcing a provider via builder
.with_execution_provider(ExecutionProvider::Cuda)

  6. How the next-plaid-client architecture works

    main

    The SDK provides two primary client types built on a shared BaseNextPlaidClient that handles URL construction, payload preparation, response parsing, and error handling.

    • NextPlaidClient: A synchronous client using httpx.Client for blocking I/O.
    • AsyncNextPlaidClient: An asynchronous client using httpx.AsyncClient for asyncio-based I/O.

    The SDK uses dataclass-based models for type-safe requests and responses, and features an exception hierarchy for structured error handling.

  7. Search modes in ColGREP

    main

    ColGREP offers four distinct ways to find code:

    1. Semantic + Keyword Hybrid (Default): Combines ColBERT semantic search with FTS5 trigram keyword search using Reciprocal Rank Fusion (RRF). This allows finding code by meaning (e.g., "database connection") and by exact identifiers or substrings (e.g., "parse_arguments").
    2. Regex Search: Uses the -e flag for traditional pattern matching (ERE syntax).
    3. Regex + Semantic Hybrid: Uses regex to narrow down candidates and semantic search to rank them. For example, finding all async fn declarations and ranking them by their relevance to "error handling".
    4. Pure Semantic Search: Disables keyword fusion to search only by meaning. Use --semantic-only for a single query or colgrep settings --no-hybrid-search to make it persistent.
    # Hybrid (Default)
colgrep "database connection pooling"

# Regex
colgrep -e "async fn\s+\\w+"

# Regex + Semantic Hybrid
colgrep -e "async fn" "error handling"

# Pure Semantic (One-shot)
colgrep --semantic-only "error handling"

  8. How ColGREP works: The search pipeline

    main

    ColGREP uses a multi-stage pipeline to provide semantic and keyword-based code search:

    1. Parsing: Uses Tree-sitter to extract code units (functions, methods, classes, etc.) for 100% coverage.
    2. Analysis: Enriches units with 5 layers of metadata: AST (signatures/params), Call Graph (calls/called_by), Control Flow (loops/error handling), Data Flow (variables), and Dependencies (imports).
    3. Structured Text: Converts units into a rich text representation (including signatures, descriptions, and calls) to provide better signal for the model.
    4. Encoding: Uses the ColBERT model (default: LateOn-Code-edge) to produce multi-vector embeddings, allowing fine-grained token-level matching.
    5. Indexing: Uses the PLAID algorithm with product quantization and memory-mapping for fast, compressed, and incremental storage.
    6. Search: Executes a hybrid search combining:
      • Metadata Pre-filtering: Using SQLite (e.g., via --include or --exclude).
      • Regex Filtering: If -e is provided.
      • Semantic Ranking: Via ColBERT MaxSim scoring.
      • Keyword Search: Via FTS5 BM25 on metadata.
      • RRF Fusion: Merges semantic and keyword results (default alpha=0.75 favoring semantic).

  9. How NextPlaid's multi-vector search works

    main

    NextPlaid implements multi-vector search (ColBERT style), which preserves fine-grained information by keeping one embedding per token (e.g., ~300 vectors per document) rather than collapsing a document into a single vector.

    At query time, it uses MaxSim scoring: each query token finds its best match across all document tokens. To manage the increased storage requirements, NextPlaid uses product quantization (2-bit or 4-bit) and memory-mapped indices, allowing large collections to run efficiently on a single machine.

  10. Understand the exported model structure

    main

    When a model is exported, it is organized into a directory named after the model. The structure is as follows:

    models/<model-name>/
├── model.onnx                        # FP32 ONNX model
├── model_int8.onnx                   # INT8 quantized (created by default)
├── tokenizer.json                    # HuggingFace fast tokenizer
└── onnx_config.json                  # Model configuration

  11. Understand ColGrep's local-first privacy model

    main

    ColGrep is designed as a local-first semantic code search tool. To ensure privacy, all core operations are performed on your local machine:

    • Indexing & Search: All indexing and search processes happen locally. No source code, embeddings, or search queries are transmitted to external servers.
    • Data Storage: Search indices are stored locally within the .colgrep/ directory inside your project folder and are never uploaded or shared.
    • Source Code Access: While ColGrep reads your local source files to build the index, that data remains on your machine.
    • Telemetry: ColGrep does not collect telemetry, usage data, analytics, or crash reports.

  12. Understand the Index File Structure

    main

    A next-plaid index is stored as a directory containing several files that manage quantization, clustering, and metadata. When loading an index, chunk-specific files (e.g., 0.codes.npy, 0.residuals.npy) are automatically merged into merged_codes.npy and merged_residuals.npy for memory-mapped access.

    Core Index Files

    • metadata.json: Contains index metadata like num_docs, nbits, and partitions.
    • centroids.npy: Centroid embeddings of shape [K, dim].
    • avg_residual.npy: Average residual per dimension.
    • bucket_cutoffs.npy & bucket_weights.npy: Quantization boundaries and reconstruction values.
    • cluster_threshold.npy: Threshold for outlier detection.
    • ivf.npy & ivf_lengths.npy: The inverted file (doc IDs per centroid) and the length of each posting list.
    • plan.json: The indexing plan.
    • merged_codes.npy & merged_residuals.npy: Memory-mapped centroid codes and quantized residuals.
    • metadata.db: An optional SQLite database for metadata storage.