DOCSEARCHbyAlgolia
Submit Docs

PyTerrier

repository·master·Indexed 19 days ago

https://github.com/terrier-org/pyterrier

A Python library for building and experimenting with information retrieval pipelines. It supports sparse, learned sparse, and dense retrieval, as well as RAG (Retrieval Augmented Generation) and re-ranking workflows. PyTerrier uses a pipeline pattern where components (Transformers) are composed using the >> operator and provides tools for conducting experiments via pt.Experiment, managing IR artifacts through HuggingFace, Zenodo, and P2P, and accessing standard datasets via ir_datasets.

Tokens
78.1K
Snippets
274
Records
353
Agent score
67%

What's inside pyterrier

  1. Overview of PyTerrier

    master

    PyTerrier is a Python framework designed for Information Retrieval (IR) research and experimentation. It provides a composable environment to build IR pipelines, covering the full lifecycle of retrieval, reranking, and answering.

    Key capabilities include:

    • State-of-the-Art IR Methods: Support for advanced techniques like Adaptive Retrieval and RankZephyr.
    • Extensibility: A common data model allows users to construct pipelines by combining different operators.
    • Experimentation: Built-in support for conducting IR experiments using hundreds of datasets and dozens of evaluation measures.
    • Diverse Retrieval Engines: Beyond the core Terrier engine, PyTerrier supports PISA, Anserini, FAISS, BMP, and various external search APIs.

  2. Importing Datasets with the datasets module

    master

    The pyterrier.datasets module provides easy access to standard test collections (notably from TREC). Each defined dataset can provide:

    • Corpus files: The documents making up the corpus.
    • Topics (queries): A Pandas DataFrame ready for retrieval tasks.
    • Relevance assessments (qrels): A Pandas DataFrame ready for evaluation.
    • Terrier indices: Ready-made indices where available.

    Key functions include:

    • pyterrier.datasets.list_datasets(): Lists available datasets.
    • pyterrier.datasets.find_datasets(): Searches for datasets.
    • pyterrier.datasets.get_dataset(name): Retrieves a specific dataset object.
    • pyterrier.datasets.Dataset: The class representing a dataset.

  3. Use flake8 extensions for PyTerrier validation

    master
    PyTerrier provides a flake8 extension designed to enforce specific coding standards related to its Java integration. Specifically, the JavaCheck rule ensures that the @pt.java.required annotation is correctly applied to methods or components that require it. If the annotation is missing where required, the linter will raise error code PT100.

  4. Explore PyTerrier Extensions

    master

    PyTerrier supports various specialized plugins for advanced search tasks:

    • Pyterrier_DR: Single-representation dense retrieval.
    • Pyterrier_RAG: Retrieval augmented generation and LLM access.
    • Pyterrier_ColBERT2: Multiple-representation dense retrieval and/or neural reranking.
    • Pyterrier_PISA: Fast in-memory indexing and retrieval using PISA.
    • Pyterrier_T5: Neural reranking (monoT5, duoT5).
    • Pyterrier_GenRank: Generative listwise reranking (RankVicuna, RankZephyr).
    • Pyterrier_doc2query: Neural augmented indexing.
    • Pyterrier_SPLADE: Neural augmented indexing.

  5. Integrate Terrier with PyTerrier

    master
    PyTerrier provides an integration with the Terrier open-source search engine (developed at the University of Glasgow). This integration allows you to leverage Terrier's core inverted indexing, retrieval functionality, built-in retrieval models, and query expansion techniques directly within your Python workflows. The integration utilizes the Terrier Java package.

  7. Functionalities of Terrier-Python-Helper

    master

    The Terrier-Python-Helper package provides several utility Java classes to improve the integration between the Terrier engine and Python:

    • Stdout/Stderr Redirection: Redirects Java stdout and stderr to match the Python process output.
    • Logging Control: Allows changing the logging level.
    • Java Collection Utilities: Provides a Collection class optimized for use from Java.
    • Multi-threaded Indexing: Supports multi-threaded indexing operations.

  8. Read and write files with pt.io

    master
    The pyterrier.io module provides utility methods for file I/O operations. It includes specialized support for reading and writing standard information retrieval formats, such as TREC-formatted topics files and run files (query results).

  10. Apply query expansion to retrieval pipelines

    master

    PyTerrier allows you to compose retrieval pipelines using the >> operator to apply query expansion techniques. This allows you to wrap a base retriever (like BM25 or DPH) with a rewriting step and then pass the expanded query back into the retriever.

    Available query expansion modules in pt.rewrite:

    • Bo1QueryExpansion
    • KLQueryExpansion
    • RM3 (uses default parameters: 10 expansion terms, 3 documents, and lambda=0.5)

    Example pipeline: BM25 >> RM3 >> BM25 (Apply RM3 expansion to a BM25 retriever).

    Bo1 = pt.rewrite.Bo1QueryExpansion(index)
KL = pt.rewrite.KLQueryExpansion(index)
RM3 = pt.rewrite.RM3(index)

# Example: Evaluating BM25 with various expansion methods
pt.Experiment(
    [
            BM25, 
            BM25 >> Bo1 >> BM25, 
            BM25 >> KL >> BM25, 
            BM25 >> RM3 >> BM25, 
    ],
    pt.get_dataset("trec-robust-2004").get_topics(),
    pt.get_dataset("trec-robust-2004").get_qrels(),
    eval_metrics=["map", "P_10", "P_20", "ndcg_cut_20"],
    names=["BM25", "+Bo1", "+KL", "+RM3"]
    )

  11. Perform Set Intersection (`&`) and Union (`|`) on document sets

    master

    Use set operators to combine the document sets returned by two transformers. Note that these operators do not return scores or ranks; they only return the resulting document sets (the documents are effectively re-scored/re-ranked as a set).

    • & (Intersection): Returns only the documents that appear in both retrieval sets.
    • | (Union): Returns all documents that appear in either retrieval set.
    BM25 = pt.terrier.Retriever(index, "BM25")
PL2 = pt.terrier.Retriever(index, "PL2")

# Intersection: documents in both
res_intersection = (BM25 & PL2).transform(topics)

# Union: documents in either
res_union = (BM25 | PL2).transform(topics)