model2vec

Model2Vec provides models specifically optimized for retrieval tasks.

Performance comparison (Retrieval Score):

• all-MiniLM-L6-v2 (Transformer baseline): 42.92
• potion-retrieval-32M (Optimized static): 35.06
• static-retrieval-mrl-en-v1: 34.95
• potion-base-32M (General-purpose static): 32.67

potion-retrieval-32M is the most performant static retrieval model, reaching 81.69% of the performance of the transformer-based all-MiniLM-L6-v2.

Multilingual performance is measured using the MMTEB leaderboard, which uses a Borda count over per-task ranks to reward consistency across different task types.

Key findings:

• potion-multilingual-128M is the most performant static multilingual model, reaching 90.86% of the performance of LaBSE.
• potion-multilingual-128M supports 101 languages.
• static-similarity-mrl-multilingual-v1 is better suited for retrieval and STS tasks, while potion-multilingual-128M excels at classification and clustering.

Task Abbreviations used in MMTEB results:

• BitMining: Bitext Mining
• Class: Classification
• Clust: Clustering
• InstRet: Instruction Retrieval
• MultiClass: Multilabel Classification
• PairClass: PairClassification
• Rank: Reranking
• Ret: Retrieval
• STS: Semantic Textual Similarity

Model2Vec models are evaluated using the Massive Text Embedding Benchmark (MTEB), as well as PEARL (phrase representation) and WordSim (word similarity).

Key findings:

• potion-base-32M is the most performant static embedding model, achieving 93.21% of the performance of all-MiniLM-L6-v2 while being significantly faster.
• The potion and M2V models are categorized as static models.

Task Abbreviations used in MTEB results:

• Class: Classification
• Clust: Clustering
• PairClass: PairClassification
• Rank: Reranking
• Ret: Retrieval
• STS: Semantic Textual Similarity
• Sum: Summarization

The StaticModelForClassification is designed to be extensible. If you need to modify the model behavior (e.g., adding LayerNorm), you can subclass StaticModelForClassification and override specific internal functions:

• construct_head: The primary function to update if you want to change the classifier head architecture.
• train_test_split: Governs the data split before classification.
• prepare_dataset: Selects the torch.Dataset used by the Dataloader.
• _encode: The encoding function used in the model.
• fit: Contains the lightning-related fitting logic.

Model2Vec creates fast static embedding models without requiring training data. The process involves:

1. Passing a vocabulary through a Sentence Transformer model.
2. Reducing the dimensionality of the resulting embeddings using PCA.
3. Weighting the embeddings using SIF (Smooth Inverse Frequency) weighting.

During inference, the model computes the embedding of a sentence by taking the mean of all token embeddings present in that sentence.

To use a trained classifier in inference-only environments without requiring torch, convert it to a scikit-learn compatible pipeline using .to_pipeline().

Persistence Options:

• Local: Use joblib or pickle to save the pipeline object.
• Hugging Face Hub: Use .save_pretrained(path) and .push_to_hub("repo_name").

Loading: To load a persisted pipeline, use StaticModelPipeline.from_pretrained("path_or_repo"). Loading from disk is extremely fast (~30ms).

The model2vec.inference subpackage provides helper functions for running inference using trained models that have been exported as scikit-learn compatible pipelines.

You can use StaticModelPipeline.from_pretrained to load a pre-trained model from the HuggingFace Hub and then use the .predict() method to perform inference on text strings.

from model2vec.inference import StaticModelPipeline

# Load a pre-trained classifier from HuggingFace
classifier = StaticModelPipeline.from_pretrained("minishlab/potion-8m-edu-classifier")

# Perform inference on a text string
label = classifier.predict("Attitudes towards cattle in the Alps: a study in letting go.")

To use the training features (including StaticModelForClassification), you must install the [train] extra for model2vec.

pip install model2vec[train]

Model2Vec provides several self-contained IPython notebooks to demonstrate its capabilities. These tutorials cover semantic search, text chunking, and classification training.

Available tutorials include:

• Recipe search: Demonstrates lightning-fast semantic search by distilling a small model. It compares tiny models against larger ones and introduces the Fattoush concept.
• Semantic chunking: Shows how to chunk text into meaningful segments using Chonkie and efficiently query those chunks with Vicinity.
• Training a classifier: Demonstrates how to train a high-performance classifier using model2vec, which is particularly effective for small datasets.

To use the inference capabilities of model2vec, install the package with the [inference] extra using pip:

pip install model2vec[inference]

The model2vec library provides the fastest and most lightweight way to run Model2Vec models. Use the StaticModel.from_pretrained method to load a model and the .encode() method to compute embeddings for a list of strings.

from model2vec import StaticModel

# Load a pretrained Model2Vec model
model = StaticModel.from_pretrained("{{ model_name }}")

# Compute text embeddings
embeddings = model.encode(["Example sentence"])

You can fine-tune classification models on top of Model2Vec models or pre-trained models. This supports both single-label and multi-label classification datasets.

Note: This requires the model2vec[train] extra.

import numpy as np
from datasets import load_dataset
from model2vec.train import StaticModelForClassification

# Initialize a classifier from a pre-trained model
classifier = StaticModelForClassification.from_pretrained(model_name="minishlab/potion-base-32M")

# Load a dataset (supports single and multi-label)
ds = load_dataset("setfit/subj")

# Train the classifier on text (X) and labels (y)
classifier.fit(ds["train"]["text"], ds["train"]["label"])

# Evaluate the classifier
classification_report = classifier.evaluate(ds["test"]["text"], ds["test"]["label"])