rerankers

The .rank() method returns a RankedResults object. This object contains a list of Result objects.

Key methods and properties:

• top_k(k): Returns the top k Result objects.
• results: The full list of Result objects.
• query: The original query used.

Each Result object contains:

• document: A Document object containing the text, doc_id, and metadata.
• score: The reranking score.
• rank: The integer rank.

You can access document properties directly from a Result object (e.g., result.text or result.metadata).

The rerankers library provides a unified interface for various reranking strategies. Depending on your needs, you can use different categories of rerankers:

• Transformers-based: Cross-encoders, T5, and ColBERT.
• RankGPT: Listwise reranking.
• API-based: Cohere, Jina, MixedBread, Pinecone, Isaacus, and HuggingFace's Text-Embedding-Server (TEI).
• FlashRank: ONNX-optimised rerankers that are very fast on CPU.
• RankLLM: Improved RankGPT with support for local models like RankZephyr and RankVicuna (requires Python 3.10+).
• Multi-Modal: Support for models like MonoQwen2-VL (requires flash-attention, peft, accelerate, and recent transformers).

The rerankers core package is lightweight and ships with no dependencies by default. You can install specific extras to support different model architectures:

• Core only: pip install rerankers
• Transformers-based (cross-encoders, t5, colbert): pip install "rerankers[transformers]"
• RankGPT: pip install "rerankers[gpt]"
• API-based (Cohere, Jina, MixedBread, Pinecone, Isaacus): pip install "rerankers[api]"
• FlashRank (ONNX-optimised, fast on CPU): pip install "rerankers[flashrank]"
• RankLLM (Requires Python 3.10+): pip install "rerankers[rankllm]"
• Multi-Modal (MonoVLM models): pip install "rerankers[monovlm]"
• LLM-Layerwise: pip install "rerankers[llmlayerwise]"
• All extras: pip install "rerankers[all]"

The .rank() method returns a RankedResults object, which is a collection of Result objects.

Key features of RankedResults:

• top_k(k): Returns the top k results, sorted by rank.
• has_scores: A boolean attribute indicating if the model provided numerical scores.
• get_score_by_docid(doc_id): Retrieves the score for a specific document ID.
• query: Contains the original query used for the ranking.

Key features of Result:

• text: The original document text.
• doc_id: The identifier for the document.
• rank: The 1-indexed rank of the document.
• score: The numerical score returned by the model (if available).

Cross-encoders take both the query and document as input to capture subtle interactions. rerankers provides balanced defaults for this model family.

To load a specific model from the Hugging Face Hub, provide the model path and specify model_type="cross-encoder" to avoid warnings.

Available Arguments for Cross-Encoders:

• model_type: Set to "cross-encoder" when loading specific Hub models.
• lang: Set to a 2-letter ISO language code (e.g., 'fr') to attempt loading a language-specific default model.
• verbose: Controls logging (0 to suppress most messages).
• dtype: The data type for the model (e.g., None for auto-detection of fp16/fp32).
• device: The device to use (cuda, mps, or cpu).
• batch_size: Number of documents to process in a batch (default: 16).

T5-based rerankers use a SequenceToSequence approach, predicting relevance tokens.

Arguments for T5 Models:

• model_type: Set to "t5" when loading specific T5 models.
• token_false: The token representing non-relevance (default: "auto").
• token_true: The token representing relevance (default: "auto").
• return_logits: Whether to return a normalized score or the raw logit for token_true (default: False).

# Load default T5
ranker = Reranker('t5')

# Load specific T5 model
ranker = Reranker("unicamp-dl/ptt5-base-pt-msmarco-10k-v2", model_type='t5', verbose=0)

ColBERT uses a late-interaction approach. While it behaves like a bi-encoder during document encoding, it acts as a powerful reranker during inference.

Arguments for ColBERT Models:

• model_type: Set to "colbert" when loading specific ColBERT models.
• query_token: The token prepended to queries (e.g., "[unused0]").
• document_token: The token prepended to documents (e.g., "[unused1]").

# Load default ColBERT
ranker = Reranker('colbert')

# Load specific ColBERT model
ranker = Reranker('antoinelouis/colbertv2-camembert-L4-mmarcoFR', model_type="colbert")

FlashRank provides ONNX-optimized weights designed for very fast inference on CPUs.

# Load default FlashRank model (MiniLM-L12-v2)
ranker = Reranker('flashrank')

# Load a specific FlashRank model
ranker = Reranker('ms-marco-TinyBERT-L-2-v2', model_type='flashrank')

from rerankers import Reranker

# Load default FlashRank model
ranker = Reranker('flashrank')

# Load a specific FlashRank model
ranker = Reranker('ms-marco-TinyBERT-L-2-v2', model_type='flashrank')

To use multi-modal reranking (e.g., for images), use the monovlm model type. It is highly recommended to run this on a GPU.

To install the necessary dependencies for multi-modal support, use:

pip install rerankers[monovlm]

By default, flash-attention is enabled to manage high memory consumption. If you need to use a different attention implementation, you can specify it via the attention_implementation parameter.

from rerankers import Reranker

ranker = Reranker("monovlm", device='cuda')