RAGLite

A high-quality manual pipeline involves: searching for a large set of chunks (e.g., 20), reranking them to select the top 5, grouping them into chunk spans, and then generating the response. This provides better precision than simple retrieval.

from raglite import ( 
    hybrid_search, keyword_search, vector_search, 
    retrieve_chunks, rerank_chunks, retrieve_chunk_spans, 
    add_context, rag 
)

user_prompt = "How is intelligence measured?"

# 1. Search (Hybrid)
chunk_ids_hybrid, _ = hybrid_search(user_prompt, num_results=20, metadata_filter={"topic": "physics"}, config=my_config)

# 2. Retrieve
chunks_hybrid = retrieve_chunks(chunk_ids_hybrid, config=my_config)

# 3. Rerank
chunks_reranked = rerank_chunks(user_prompt, chunks_hybrid, config=my_config)[:5]

# 4. Group into spans
chunk_spans = retrieve_chunk_spans(chunks_reranked, config=my_config)

# 5. Generate
messages = [add_context(user_prompt=user_prompt, context=chunk_spans, config=my_config)]
stream = rag(messages, config=my_config)
for update in stream:
    print(update, end="")

RAGLite uses a cost-minimization approach to split text into chunks (chunklets) that are optimized for retrieval and LLM processing.

The Cost Model

The splitting logic solves a dynamic programming problem where the goal is to minimize the total cost of all chunklets. The cost of a single chunklet is the sum of two components:

1. Boundary Cost: Penalizes chunklets that start or end poorly relative to Markdown structure. It uses markdown_chunklet_boundaries to assign probabilities to sentences based on whether they represent Markdown elements like heading_open, blockquote_open, or paragraph_open.
2. Statement Cost: Penalizes chunklets that deviate from an ideal information density. A 'statement' is a normalized measure of a sentence's word count. The default cost function (s - 3)² / sqrt(s) / 2 heavily penalizes chunklets that do not contain approximately 3 statements.

Optimization Constraints

• Max Size: A hard constraint on the character length of any single chunklet.
• Markdown Awareness: The algorithm prefers splitting at natural structural boundaries (like headings) to ensure semantic coherence.

To benchmark RAG performance using TREC runs, you can implement a custom evaluator by subclassing IREvaluator. The IREvaluator base class provides the infrastructure for managing dataset-specific IDs, file paths for TREC runs, and a score() method that automates the query-and-save process.

To create a functional evaluator, you must implement the following abstract methods:

1. insert_documents(max_workers: int | None = None): Logic to ingest the dataset's documents into your specific search index or vector store.
2. search(query_id: str, query: str, *, num_results: int = 10) -> list[ScoredDoc]: Logic to perform a search for a given query and return a list of ScoredDoc objects (containing query_id, doc_id, and score).

The score() method will automatically handle the creation of a .trec file in the user's data directory, iterating through the dataset queries and writing results in the standard TREC format.

from ir_datasets.datasets.base import Dataset
from ir_measures import ScoredDoc
# You must implement this class
class MyCustomEvaluator(IREvaluator):
    def insert_documents(self, max_workers: int | None = None) -> None:
        # Your implementation
        pass

    def search(self, query_id: str, query: str, *, num_results: int = 10) -> list[ScoredDoc]:
        # Your implementation
        return []

RAGLite implements an agentic retrieval pattern. When rag() or async_rag() is called:

1. Initial Attempt: The LLM receives the user prompt. If the prompt doesn't contain explicit RAG context, the system provides a search_knowledge_base tool.
2. Tool Call: If the LLM determines it needs more information, it calls search_knowledge_base with a precise, single-faceted query.
3. Iterative Search: The system executes the search and provides the results back to the LLM. The LLM can then decide to call the tool again with a refined query if necessary (up to config.agentic_iterations).
4. Novelty Check: To prevent context bloat, RAGLite only keeps ChunkSpan objects that introduce at least one new chunk ID not seen in previous iterations.
5. Final Answer: Once the LLM has sufficient information, it responds. If tool calls were made, a follow-up generation is triggered with a NO_TOOLS_FOLLOW_UP_PROMPT to ensure the LLM focuses on answering rather than searching again.