LEANN Vector Database

LEANN provides a high-performance vector search and embedding pipeline with the following core capabilities:

• Real-time Embeddings: Uses optimized ZMQ servers and a highly optimized search paradigm (overlapping and batching) to compute embeddings dynamically, eliminating the need for heavy embedding storage.
• AST-Aware Code Chunking: Performs intelligent code chunking that preserves semantic boundaries (such as functions, classes, and methods) for Python, Java, C#, and TypeScript files.
• Scalable Architecture: Designed to handle millions of documents even on consumer-grade hardware.
• Graph Pruning: Employs advanced techniques to minimize the storage footprint of vector search.
• Pluggable Backends: Supports multiple search backends, including HNSW/FAISS (default) and DiskANN for large-scale deployments.

The LEANN-RAG evaluation dataset is organized into three functional components required for recall evaluation:

1. Pre-built LEANN Indices: Located in dpr/ and rpj_wiki/. These indices are built using leann-core and are consumed by the LeannSearcher.
2. Ground Truth Data: Located in ground_truth/. Contains files like flat_results_nq_k3.json which map queries to original passage IDs (evaluated using the Contriever model).
3. Queries: Located in queries/. Contains the nq_open.jsonl file containing the Natural Questions queries used for evaluation.

LEANN achieves high compression (up to 98% reduction in storage) by storing a pruned neighbor graph instead of full embedding vectors.

Key Mechanisms:

• Storage: Instead of storing heavy vectors (which can take GBs for large datasets), it stores a graph structure.
• Search: During a search query, embeddings are recomputed on-demand via a local daemon.
• Latency Management: It leverages OpenClaw's async "sleep time compute" model, meaning the recomputation latency occurs during idle periods and is invisible to the user.

Prompt templates should be used ONLY with task-specific embedding models (e.g., Google's EmbeddingGemma) that are trained to distinguish between document and query contexts.

Warning: Do NOT use prompt templates with regular models like nomic-embed-text, text-embedding-3-small, or bge-base-en-v1.5. Adding prompts to these models will corrupt the resulting embeddings.

To use templates, provide a different --embedding-prompt-template for the build command (document context) and the search command (query context).

# Build with document prompt
leann build my-docs --embedding-prompt-template "title: none | text: "

# Search with query prompt
leann search my-docs --query "your question" --embedding-prompt-template "task: search result | query: "

The FinanceBench benchmark uses two distinct evaluation methodologies:

Retrieval Evaluation

Matches retrieved documents against ground truth using three strategies:

1. Exact text overlap: Direct substring matches.
2. Number matching: Matches key financial figures (e.g., $1,577, 1.2B).
3. Semantic similarity: Word overlap with a 20% threshold.

QA Evaluation

Uses an LLM (typically GPT-4o) to evaluate the correctness of generated answers. This method is designed to be robust by:

• Handling numerical rounding and equivalent representations.
• Considering fractions, percentages, and decimal equivalents.
• Evaluating semantic meaning rather than requiring an exact text match.

Choose a backend based on your hardware and storage requirements:

LEANN is designed as a low-storage vector index. Unlike traditional vector databases that store every single embedding, LEANN uses several core techniques to reduce footprint:

• Graph-based selective recomputation: Instead of storing all embeddings, LEANN stores a pruned graph structure and recomputes embeddings only for nodes in the search path.
• High-degree preserving pruning: It keeps important "hub" nodes while removing redundant connections to save space.
• Two-level search: A smart graph traversal that prioritizes promising nodes.

Supported Backends

• HNSW (default): Best for most datasets; achieves maximum storage savings through full recomputation.
• DiskANN: Offers superior search performance using PQ-based graph traversal with real-time reranking for a better speed-accuracy trade-off.

You can trade storage space for lower latency by disabling recomputation. This stores full embeddings, avoiding the need to recompute them at query time. This is ideal for high QPS or interactive assistants.

Trade-offs:

• Pros: Significantly lower latency; no network/compute hops for embeddings at query time.
• Cons: Much higher storage requirements (10–100× more); larger memory footprint.

Build without recomputation (Note: HNSW requires --no-compact in this mode):

leann build my-index --no-recompute --no-compact

Search without recomputation:

leann search my-index "your query" --no-recompute

Python API usage:

from leann import LeannSearcher

searcher = LeannSearcher("/path/to/my-index.leann")
results = searcher.search("your query", top_k=10, recompute_embeddings=False)

The flashlib_ivf backend persists the IVF-Flat index as a set of torch tensors (including centroids, cell-contiguous data, row IDs, and CSR offsets).

When an index is built, it creates:

1. <index>.flashlib_ivf.pt: The serialized torch tensors.
2. <index>.flashlib_ivf_id_map.json: The ID map.

Upon starting a LeannSearcher, these files are reloaded directly onto the GPU, avoiding the need for k-means re-training.