SymSpell Documentation

SymSpell is a high-performance spelling correction and fuzzy search library based on the Symmetric Delete algorithm. It is designed for speed, being significantly faster than standard approaches (like Norvig's algorithm or BK-trees) by using a delete-only edit candidate generation and pre-calculation strategy.

Key capabilities include:

• Single word spelling correction: Fast lookup for individual terms.
• Compound aware multi-word spelling correction: Handles multi-word strings by addressing compound splitting, decompounding, and automatic correction of long text.
• Word Segmentation: Useful for processing noisy text.

You can add SymSpell to your project in three ways:

1. Manual Integration: Copy SymSpell.cs, EditDistance.cs, and a frequency dictionary (e.g., frequency_dictionary_en_82_765.txt) directly into your project files.
   • Tip: Enabling the compiler option "Prefer 32-bit" significantly reduces memory consumption of the precalculated dictionary.
2. NuGet for .NET Framework: Install via the NuGet Package Manager. The frequency dictionary is automatically included.
3. NuGet for .NET Core: Install via the NuGet Package Manager. You must manually copy the frequency dictionary file to your project.

SymSpell targets .NET Standard v2.0 and is compatible with .NET Framework, .NET Core, and XAMARIN (iOS, OS X, Android).

The repository includes a Python script designed to automate the creation of custom frequency dictionaries. This script handles:

1. Downloading the Ngram dataset.
2. Decompressing the dataset.
3. Merging the Ngram data with existing wordlists to create a unified frequency dictionary.

SymSpell uses plain text files in UTF-8 encoding for dictionaries.

Format Requirements:

• Structure: Each line contains a word-frequency pair.
• Separators: Word and frequency are separated by a space or tab.
• Columns: By default, the word is in the first column and frequency in the second. You can customize this using termIndex and countIndex in LoadDictionary() or LoadBigramDictionary().
• Case: Both dictionary terms and input terms are expected to be in lower case.
• Bigrams: When using LoadBigramDictionary, if no separator is specified, it expects two term parts (a bigram).

To correct single words, initialize a SymSpell object, load a frequency dictionary, and use the Lookup method.

Parameters for Lookup:

• inputTerm: The word to check.
• suggestionVerbosity: Determines how many suggestions are returned. Use SymSpell.Verbosity.Closest, Top, or All.
• maxEditDistanceLookup: The maximum edit distance allowed for the lookup (must be $\le$ maxEditDistanceDictionary used during initialization).

The WordSegmentation method divides a string into words by inserting missing spaces. It is designed for noisy text where spaces might be missing (e.g., OCR errors, URLs, or typing errors).

Key Features:

• Corrects misspelled words during segmentation.
• Considers existing spaces for optimum segmentation.
• Uses a Triangular Matrix approach for $O(n)$ linear runtime.

Returns: A result object containing correctedString and distanceSum.

Use the Lookup method for fast spelling correction of single words. You can control the results using a Verbosity parameter and a Maximum edit distance parameter.

Verbosity Levels

• Top: Returns only the top suggestion with the highest term frequency among those with the smallest edit distance found.
• Closest: Returns all suggestions of the smallest edit distance found, ordered by term frequency.
• All: Returns all suggestions within the maxEditDistance, ordered by edit distance and then by term frequency.

Dictionary Setup

You must provide a word frequency dictionary. This can be done in two ways:

1. LoadDictionary: Directly load a dictionary from text files.
2. CreateDictionary: Generate a dictionary from a large text corpus.

Use LookupCompound to perform automatic spelling correction on multi-word input strings. Unlike Lookup, which treats every input as a single term, LookupCompound handles:

1. Compound splitting & decompounding:
   • Correcting mistakenly inserted spaces within a single word.
   • Correcting mistakenly omitted spaces between two words.
   • Handling multiple input terms with or without spelling errors.
2. Automatic spelling correction:
   • Makes educated choices for automatic correction of long text strings where manual selection is infeasible.

It can handle mixed error types including splitting, concatenation, substitution, transposition, deletion, and insertion errors within the same sequence.

To support compound splitting and merging (e.g., correcting whereis th elove to where is the love), you must first load a bigram dictionary using LoadBigramDictionary. Then, use LookupCompound to find suggestions for multi-word strings.

Parameters for LookupCompound:

• inputTerm: The multi-word string.
• maxEditDistanceLookup: The maximum edit distance allowed per single word (not for the whole string).