symspellpy Documentation

To use a custom algorithm for calculating string distances in symspellpy, you must create a class that inherits from AbstractDistanceComparer and implements the distance(self, string_1, string_2, max_distance) method.

Once implemented, you pass this comparer to the Editdistance class using the DistanceAlgorithm.USER_PROVIDED flag. Finally, initialize SymSpell with this custom Editdistance instance as the distance_comparer argument.

import importlib.resources
from symspellpy import SymSpell
from symspellpy.abstract_distance_comparer import AbstractDistanceComparer
from symspellpy.editdistance import DistanceAlgorithm, Editdistance

class CustomComparer(AbstractDistanceComparer):
    def distance(self, string_1, string_2, max_distance):
        # Implement your custom logic here
        # Example: return distance if within max_distance, else -1
        distance = ... 
        return -1 if distance > max_distance else distance

# 1. Wrap the custom comparer in Editdistance with USER_PROVIDED algorithm
custom_comparer = Editdistance(DistanceAlgorithm.USER_PROVIDED, CustomComparer())

# 2. Inject the custom comparer into SymSpell
sym_spell = SymSpell(distance_comparer=custom_comparer)

If you need a specific distance metric not provided by the library, you can implement your own by subclassing AbstractDistanceComparer and implementing the distance(string_1, string_2, max_distance) method. Then, use DistanceAlgorithm.USER_PROVIDED to inject it into EditDistance.

Note: If you pass a comparer to EditDistance but do not set the algorithm to USER_PROVIDED, a warning will be issued and your comparer will be ignored in favor of the built-in algorithm.

from symspellpy.editdistance import EditDistance, DistanceAlgorithm
from symspellpy.abstract_distance_comparer import AbstractDistanceComparer

class MyCustomComparer(AbstractDistanceComparer):
    def distance(self, string_1: str, string_2: str, max_distance: int) -> int:
        # Your custom logic here
        return 0

custom_comparer = MyCustomComparer()
edist = EditDistance(DistanceAlgorithm.USER_PROVIDED, comparer=custom_comparer)
print(edist.compare("a", "b", 1))

You can use the word_segmentation method to split a continuous string of characters (a sentence without spaces) into individual words.

To ensure the segmentation process only splits words and does not attempt to correct spelling, initialize SymSpell with max_dictionary_edit_distance=0.

When loading a dictionary via load_dictionary, you must specify the term_index (the column containing the word) and the count_index (the column containing the term frequency).

import importlib.resources
from symspellpy.symspellpy import SymSpell

# Initialize with max_dictionary_edit_distance=0 to avoid spelling correction
sym_spell = SymSpell(max_dictionary_edit_distance=0, prefix_length=7)

# Load the dictionary
dictionary_path = importlib.resources("symspellpy") / "frequency_dictionary_en_82_765.txt"
sym_spell.load_dictionary(dictionary_path, term_index=0, count_index=1)

# Segment a string without spaces
input_term = "thequickbrownfoxjumpsoverthelazydog"
result = sym_spell.word_segmentation(input_term)

# result contains corrected_string, distance_sum, and log_prob_sum
print(f"{result.corrected_string}, {result.distance_sum}, {result.log_prob_sum}")