DOCSEARCHbyAlgolia
Submit Docs

NeMo Retriever

repository·main·Indexed 25 days ago

https://github.com/nvidia/nemo-retriever

A scalable RAG ingestion pipeline from Nvidia for extracting and contextualizing document content, including text, tables, charts, and infographics. The library automates workflows from document splitting and OCR to embedding generation and storage in vector databases such as LanceDB. It includes a comprehensive agent evaluation framework with tools for extracting queries, running agent profiles (baseline vs. skill), and generating performance reports using LLM judges.

Tokens
110.7K
Snippets
221
Records
534
Agent score
81%

What's inside nemo-retriever

  1. Overview of Retriever Harness

    main

    The Retriever Harness is a tool for running registered ingest and retrieval benchmarks with repeatable configurations and machine-readable results. It supports two execution paths:

    1. Library execution: Runs Retriever directly in local or Ray-backed batch mode.
    2. Service execution: Tests an existing Retriever endpoint or a temporary Helm deployment.

    Both paths utilize the same runfiles, metric gates, and artifact contracts. Use retriever ingest and retriever query for standard data tasks, and retriever harness for benchmarking and evaluation.

  2. Overview of nemo-retriever Helm chart

    main

    The nemo-retriever Helm chart deploys the service mode of nemo-retriever to Kubernetes. This service is a FastAPI document ingestion server that streams uploads through NVIDIA NIM microservices (such as object detection, OCR, and VLM embed) and exposes result and status APIs via HTTP and SSE.

    The chart supports two deployment layers:

    1. The service: A FastAPI server that can be deployed as a single standalone Deployment or a split topology (gateway / realtime / batch).
    2. The NIMs: Optional, GPU-backed NIMCache and NIMService custom resources (apiVersion: apps.nvidia.com/v1alpha1) managed by the NVIDIA NIM Operator.

    If the NIM Operator is not present, the chart will still install the service, but it will fall back to using external NIM URLs provided via serviceConfig.nimEndpoints.*.

  3. Overview of Retriever CLI commands

    main

    The retriever CLI provides a workflow for document ingestion and retrieval. The primary commands are:

    • retriever ingest: Ingests supported documents and media (PDF, HTML, TXT, image, Office, audio, video) into a Retriever index.
    • retriever query: Queries a local LanceDB table created by local or batch ingest.
    • retriever query service: Queries a deployed Retriever service.
    • retriever harness run: Executes a named, code-owned benchmark.
    • retriever service: Operates a Retriever service deployment.

  4. Overview of NVIDIA NeMo Retriever

    main

    NVIDIA NeMo Retriever is a collection of microservices designed for building and scaling multimodal data extraction, embedding, and reranking pipelines. It is built with NVIDIA NIM and is part of the NVIDIA NeMo software suite.

    Key capabilities include:

    • Multimodal Data Extraction: Scalable extraction of text, tables, charts, and infographics from documents.
    • Embedding + Indexing: Embedding extracted text from chunks and images, with integration for inserting embeddings into LanceDB.
    • Retrieval: High-accuracy retrieval using semantic and hybrid search via embedding and reranking NIM microservices.

  5. Overview of NeMo Retriever Library

    main

    NVIDIA NeMo Retriever Library (NRL) is a framework for high-accuracy, scalable content and metadata extraction from various media types, including PDFs, HTML, Word docs, PowerPoint, audio, video, and images. It facilitates the extraction of text, tables, charts, infographics, and transcripts, which can then be used in generative and retrieval-augmented generation (RAG) applications.

    Key capabilities include:

    • Parallelized Extraction: Splits documents into pages and classifies sub-page content (paragraphs, tables, etc.).
    • OCR Integration: Contextualizes extracted content via optical character recognition into a standard schema.
    • Embedding Management: Computes embeddings for extracted content.
    • Vector Database Integration: Supports storing vectors in LanceDB when using vdb_op="lancedb" during upload.
    • Configurable Pipelines: Supports multiple extraction methods (e.g., pdfium or nemotron_parse for PDFs) and various pre/post-processing operations like text chunking, filtering, and image offloading.

  6. Choose a Deployment Mode

    main

    NeMo Retriever can be deployed in several ways depending on your requirements:

    • Library mode: Run without the full container stack for lightweight integration.
    • Kubernetes / Helm (self-hosted): Run the full microservices pipeline on your own infrastructure using Helm charts.
    • Notebooks: Use Jupyter notebooks for experimentation and RAG (Retrieval-Augmented Generation) demonstrations.

  7. Understand the BRIGHT benchmark for reasoning-intensive retrieval

    main

    The BRIGHT benchmark is designed to evaluate reasoning-intensive information retrieval. Unlike traditional benchmarks (e.g., BEIR, MTEB) that rely on lexical or semantic matching, BRIGHT requires multi-step reasoning to determine relevance. Relevant documents in BRIGHT often share underlying principles, theories, or algorithms with the query rather than surface-level similarity. It covers domains such as:

    • Economics
    • Mathematics
    • Programming
    • Natural Sciences

  9. Understand the QA Evaluation Pipeline data flow

    main

    The evaluation pipeline measures LLM answer quality over a RAG pipeline by retrieving context, generating answers, and scoring them against ground-truth.

    Pipeline Stages

    StageArtifacts producedCode / APIs involved
    1. Ingest + embedlancedb/<uri>/<table>/retriever ingest extracts, embeds, and writes to LanceDB. The table name must match the one used in retriever eval export (default is nemo-retriever).
    2. Optional full-page markdown indexdata/bo767_page_markdown.jsonretriever eval build-page-index uses nemo_retriever.io.markdown.build_page_index() to create an index from extraction Parquet.
    3. Retrieval exportdata/eval/bo767_retrieval_fullpage.jsonretriever eval export (via nemo_retriever.export.export_retrieval_json()) queries LanceDB. If --page-index is provided, hits are replaced with full-page markdown strings.
    4. Ground truthdata/bo767_annotations.csvQuestions/answers used for export and eval.
    5. Evaluationqa_results_*.jsonretriever eval run or operator graph chain using nemo_retriever.evaluation components: RetrievalLoaderOperator >> QAGenerationOperator >> JudgingOperator >> ScoringOperator.

    Integration Modes

    • File Mode: Evaluation reads a retrieval JSON file and a ground-truth CSV.
    • LanceDB Mode: Evaluation queries LanceDB directly (set retrieval.type: "lancedb"), skipping the export step.

  10. Understand Ingestion Jobs in NeMo Retriever

    main

    An ingestion job is the fundamental unit of work for processing input content (documents, audio, video, etc.). Jobs are not submitted as standalone JSON documents; instead, they are executed via:

    1. The ingestor Python API (e.g., using Ingestor task chains like .extract(...)).
    2. The retriever ingest CLI.

    By default, tasks are configured for strong recall. You can customize behavior using task keyword arguments (such as chunking and splitting parameters within .extract()) or by implementing custom UDF-style operations.

  11. Understand the QA Evaluation Scoring System

    main

    The QA Evaluation Pipeline uses a three-tier hierarchy to measure different layers of the RAG pipeline:

    1. Tier 1: Retrieval Quality (answer_in_context) - A programmatic check. Returns True if $\ge$ 50% of the reference answer's content words (normalized, stopwords removed) appear in the retrieved chunks. A False result indicates a retrieval failure.
    2. Tier 2: Answer Quality (Token) (token_f1) - A programmatic SQuAD-style metric measuring overlap between the generated answer and the reference using precision, recall, and f1 scores.
    3. Tier 3: Answer Quality (Semantic) (judge_score) - An LLM-as-judge metric (0.0-1.0 scale) using a dual-judge logic (bidirectional equivalence check) to determine factual correctness, allowing for paraphrasing.

    Note on Migration: The judge_score uses a new dual-judge logic. Scores from previous versions using the single-call 1-5 rubric are not comparable.