spaczz for spaCy

spaczz provides fuzzy matching and enhanced regex matching functionality for spaCy.

Key characteristics:

• API Compatibility: Components follow APIs similar to their spaCy counterparts.
• Pipeline Integration: spaczz pipeline components can be integrated into spaCy pipelines, allowing them to be saved and loaded as part of a model.
• Core Technologies: Fuzzy matching is powered by RapidFuzz, and regex matching relies on the regex library.
• Compatibility: Supports spaCy >= 3.0 and Python >= 3.7, <= 3.11.

Spaczz extends spaCy with two primary capabilities:

1. Fuzzy matching: Finding matches that are not exact string matches.
2. Multi-token regex matching: Using regular expressions to match across multiple tokens in a spaCy Doc.

Spaczz components are designed to have APIs similar to their spaCy counterparts. This allows spaczz pipeline components to integrate directly into spaCy pipelines, enabling them to be saved and loaded as part of a standard spaCy model.

You can install spaczz using pip. This library provides fuzzy matching and multi-token regex matching functionality for spaCy.

pip install spaczz

To contribute to spaczz, fork the repository and install the project and its development dependencies using Poetry. You will also need to install the en_core_web_md spaCy model separately for testing and documentation examples.

Follow these steps within the spaczz root directory:

1. Install dependencies: poetry install
2. Install pre-commit hooks: pre-commit install
3. Download the required spaCy model: poetry run python -m spacy download "en_core_web_md"

The PhraseSearcher uses a two-stage process to find high-quality phrase matches:

1. Scanning (_scan): The searcher iterates through the Doc in chunks of the same length as the query. It identifies any chunk where the match ratio is at least min_r1.
2. Optimization (_optimize): For each candidate found in the scan, the searcher attempts to "flex" the boundaries. It expands the start and end indices of the match by up to flex tokens in both directions (left, right, or both) to see if a larger or more accurate span yields a higher match ratio.

If the optimized match ratio meets the min_r2 requirement, the match is kept. Finally, filter_overlapping_matches is used to ensure the returned results do not overlap, prioritizing matches with higher ratios or earlier positions.

The SpaczzRuler is a spaCy pipeline component that adds fuzzy, regex, and token-based matches to Doc.ents. It can be used alongside statistical NER models or the standard spaCy EntityRuler to improve entity recognition accuracy. You can add it to an existing nlp object using nlp.add_pipe("spaczz_ruler") or instantiate it directly.

import spacy
from spaczz.pipeline import SpaczzRuler

nlp = spacy.blank("en")
ruler = SpaczzRuler(nlp)
ruler.add_patterns([{
    "label": "NAME", 
    "pattern": "Grant Andersen",
    "type": "fuzzy", 
    "kwargs": {"fuzzy_func": "token_sort"}
}])

# Add to pipeline
nlp.add_pipe("spaczz_ruler", config={"patterns": [...]})

doc = nlp("My name is Anderson, Grunt")
# Matches are now in doc.ents

To use spaczz with spaCy, add the spaczz_ruler to your spaCy pipeline and provide patterns. Patterns for fuzzy matching must include "type": "fuzzy".

Example setup:

import spacy
from spaczz.pipeline import SpaczzRuler

# Create a blank English model
nlp = spacy.blank("en")

# Add the spaczz_ruler to the pipeline
spaccz_ruler = nlp.add_pipe("spaczz_ruler")

# Define patterns
fuzzy_patterns = [
    {
        "label": "COUNTRY",
        "pattern": "Egypt",
        "type": "fuzzy",
        "id": "Egypt",
    }
]

# Add patterns to the ruler
spaccz_ruler.add_patterns(fuzzy_patterns)

To use the custom attributes provided by spaczz on spaCy Doc, Span, and Token objects, you must first call SpaczzAttrs.initialize(). This registers the spaczz_ prefixed extensions. If these extensions have already been registered (e.g., by another part of your application), a warning of type AttrOverwriteWarning will be issued, and the attributes will be force-overwritten.

from spaczz.customattrs import SpaczzAttrs

# Register the custom attributes on spaCy objects
SpaczzAttrs.initialize()

Because spaczz is written in pure Python and does not currently utilize spaCy vocabularies, it may be slower and more memory-intensive than native spaCy components.

To speed up FuzzyMatcher and SimilarityMatcher, you can adjust the following parameters, noting that these changes may impact match quality:

• Decrease the flex parameter towards 0.
• If flex > 0, increase the min_r1 parameter towards the value of min_r2.
• If flex > 0, lower the thresh parameter towards min_r2.

Additionally, utilizing a GPU may help speed up the matching process for SimilarityMatcher.