PolyFuzz Documentation

When you pass a single list to the .match() method instead of two separate lists, PolyFuzz performs a self-match. In this mode, the model compares the strings in the list against every other string in that same list. To prevent trivial results, PolyFuzz automatically ignores any comparison a string has with itself; otherwise, every string would simply map to itself with a perfect score.

# Example of self-matching a single list
from polyfuzz import PolyFuzz
from polyfuzz.datasets import load_company_names

data = load_company_names()
model = PolyFuzz("TF-IDF").match(data)

To perform a simple fuzzy match between two lists using the TF-IDF method, instantiate PolyFuzz with the method name and call .match(from_list, to_list). You can access the results using .get_matches().

Notes:

• To compare distances within a single list, use model.match(from_list).
• Common quick-access methods include "TF-IDF", "EditDistance", and "Embeddings" (which uses FastText).

from polyfuzz import PolyFuzz

from_list = ["apple", "apples", "appl", "recal", "house", "similarity"]
to_list = ["apple", "apples", "mouse"]

model = PolyFuzz("TF-IDF")
model.match(from_list, to_list)

# Access results
matches = model.get_matches()
print(matches)

To perform a quick comparison between two lists of strings, instantiate PolyFuzz with a matching method and call .match().

Supported quick-access methods include:

• "TF-IDF" (Character n-grams)
• "EditDistance" (Levenshtein distance)
• "Embeddings" (FastText English)

After matching, use .get_matches() to retrieve a DataFrame containing the From string, the To string, and the Similarity score.

from polyfuzz import PolyFuzz

from_list = ["apple", "apples", "appl", "recal", "house", "similarity"]
to_list = ["apple", "apples", "mouse"]

# Instantiate with method and match lists
model = PolyFuzz("TF-IDF").match(from_list, to_list)

# Access results
matches = model.get_matches()
print(matches)

To use a custom similarity or distance measure, create a class that inherits from polyfuzz.models.BaseMatcher and implements the match method.

The match method must:

1. Accept two lists of strings (from_list, to_list).
2. Return a pandas.DataFrame with exactly three columns: From, To, and Similarity.

import numpy as np
import pandas as pd
from polyfuzz.models import BaseMatcher

class MyModel(BaseMatcher):
    def match(self, from_list, to_list):
        # 1. Calculate distances/similarities
        # 2. Get best matches and scores
        # 3. Prepare dataframe
        matches = pd.DataFrame({
            'From': from_list,
            'To': mappings,
            'Similarity': scores
        })
        return matches

# Usage
custom_matcher = MyModel()
model = PolyFuzz(custom_matcher).match(from_list, to_list)

By default, PolyFuzz uses a TF-IDF implementation with single linkage to group mapped strings. However, you can replace this with any supported model or a custom model by passing it to the .group() method of a PolyFuzz instance after performing a match.

To use a different model (such as EditDistance) for grouping, instantiate the desired model and pass it to model.group().

from polyfuzz import PolyFuzz
from polyfuzz.models import EditDistance

from_list = ["apple", "apples", "appl", "recal", "house", "similarity"]
to_list = ["apple", "apples", "mouse"]

# 1. Perform the initial matching
model = PolyFuzz("TF-IDF").match(from_list, to_list)

# 2. Define a custom grouper (e.g., EditDistance)
base_edit_grouper = EditDistance(n_jobs=1)

# 3. Apply the custom grouper to the matches
model.group(base_edit_grouper)

For production environments where you need to match incoming (unseen) strings against a known set of correct strings, use the fit and transform pattern. This is more efficient than .match() because the representations (like TF-IDF) are calculated once during fit and reused during transform.

1. fit(train_words): Calculates and stores the representations for the reference list.
2. transform(unseen_words): Maps the new words to the reference list using the stored representations.

from polyfuzz import PolyFuzz

train_words = ["apple", "apples", "appl", "recal", "house", "similarity"]
unseen_words = ["apple", "apples", "mouse"]

# Fit on the reference list
model = PolyFuzz("TF-IDF")
model.fit(train_words)

# Transform incoming words
results = model.transform(unseen_words)

Install the core library via pip:

pip install polyfuzz

Depending on your requirements for language models and backends, you can install additional dependencies using the following extras:

• SBERT: pip install polyfuzz[sbert]
• Flair: pip install polyfuzz[flair]
• Gensim: pip install polyfuzz[gensim]
• SpaCy: pip install polyfuzz[spacy]
• USE (Universal Sentence Encoder): pip install polyfuzz[use]
• Fast Cosine Similarity: To speed up comparisons and decrease memory usage when using embedding models, install sparse_dot_topn via pip install polyfuzz[fast].

pip install polyfuzz
# Optional extras:
pip install polyfuzz[sbert]
pip install polyfuzz[flair]
pip install polyfuzz[gensim]
pip install polyfuzz[spacy]
pip install polyfuzz[use]
pip install polyfuzz[fast]

Install the core package via pip:

pip install polyfuzz

Depending on your required backends (transformers, embeddings, etc.), you may need to install additional dependencies:

• SBERT: pip install polyfuzz[sbert]
• Flair: pip install polyfuzz[flair]
• Gensim: pip install polyfuzz[gensim]
• Spacy: pip install polyfuzz[spacy]
• USE: pip install polyfuzz[use]

To speed up cosine similarity comparisons and reduce memory usage when using embedding models, install the fast extra:

pip install polyfuzz[fast]

Troubleshooting sparse_dot_topn: If you encounter installation issues with the fast extra, try installing it via conda first:

conda install -c conda-forge sparse_dot_topn