JamSpell Documentation

To train a custom model, you need a UTF-8 text file containing sentences (training data) and a text file containing the language alphabet.

1. Build JamSpell from source using cmake.
2. Run the ./main/jamspell train command providing the alphabet file, the training sentences file, and the desired output model filename.
3. (Optional) Evaluate the model using the evaluate/evaluate.py script.

# Training command
./main/jamspell train ../test_data/alphabet_en.txt ../test_data/sherlockholmes.txt model_sherlock.bin

# Evaluation command
python evaluate/evaluate.py -a alphabet_file.txt -jsp your_model.bin -mx 50000 your_test_data.txt

To use JamSpell in Python, you must first install swig3 via your system's package manager. Then, install the jamspell package using pip. You will need a pre-trained language model (e.g., en.bin) or a model you have trained yourself. Use jamspell.TSpellCorrector() to initialize the corrector and LoadLangModel() to load your model file.

# 1. Install swig3 (via your distro package manager)
# 2. Install jamspell
pip install jamspell

import jamspell

# Initialize and load model
corrector = jamspell.TSpellCorrector()
corrector.LoadLangModel('en.bin')

# Fix a text fragment
print(corrector.FixFragment('I am the begt spell cherken!'))
# Output: u'I am the best spell checker!'

# Get top N candidates for a list of words
print(corrector.GetCandidates(['i', 'am', 'the', 'begt', 'spell', 'cherken'], 3))

To use JamSpell in a C++ project, add the jamspell and contrib directories to your project structure. Include <jamspell/spell_corrector.hpp> and use the NJamSpell::TSpellCorrector class. The corrector requires a loaded language model via LoadLangModel().

#include <jamspell/spell_corrector.hpp>

int main(int argc, const char** argv) {
    NJamSpell::TSpellCorrector corrector;
    corrector.LoadLangModel("model.bin");

    // Fix a fragment
    corrector.FixFragment(L"I am the begt spell cherken!");

    // Get candidates
    corrector.GetCandidates({L"i", L"am", L"the", L"begt", L"spell", L"cherken"}, 3);
    
    return 0;
}

The /fix endpoint returns the corrected version of the input text using the loaded JamSpell model.

Request Methods

• GET: Expects the text as a query parameter named text.
• POST: Expects the text in the request body.

Response

• Returns the corrected text as text/plain.

Examples

GET Request: GET /fix?text=thiss is a tset

POST Request: POST /fix with body thiss is a tset

The TLangModel class is used for model lifecycle management, including training on datasets and scoring text.

• void Train(const std::string& datasetFile, const std::string& alphabetFile): Trains the model using the specified dataset and alphabet files.
• void Dump(const std::string& resultModelFile): Saves the trained model to a binary file.
• bool Load(const std::string& modelFile): Loads a previously saved model from a binary file. Returns true if successful.
• double Score(const std::wstring& wtext): Returns a score for the provided wide-string text based on the loaded model.

The TSpellCorrector class provides high-level spell correction capabilities using a loaded TLangModel.

• bool LoadLangModel(const std::string& modelFile): Loads the language model required for correction. Returns true if successful.
• std::wstring FixFragment(const std::wstring& text): Takes a wide-string fragment and returns the corrected version of that text.

The /candidates endpoint returns a JSON object containing potential spelling corrections for words in the input text that the model identifies as errors.

Request Methods

• GET: Expects the text as a query parameter named text.
• POST: Expects the text in the request body.

Response Format

The response is a JSON object with a results array. Each item in the array represents a detected error and contains:

• pos_from: The starting position of the error in the input string.
• len: The length of the error word.
• candidates: An array of up to 7 suggested replacement strings.

Example JSON Response

{
    "results": [
        {
            "pos_from": 0,
            "len": 5,
            "candidates": [
                "this",
                "thiss"
            ]
        }
    ]
}

The following table defines the command-line interface for the JamSpell tool.

Usage: <executable> mode args

| Mode | Arguments | Description |
|------|-----------|-------------|
| `train` | `alphabet.txt dataset.txt resultModel.bin` | Trains a model using the provided alphabet and dataset. |
| `score` | `model.bin` | Reads sentences from stdin and returns a score for each. |
| `correct` | `model.bin` | Reads sentences from stdin and returns the corrected text. |
| `fix` | `model.bin input.txt output.txt` | Automatically fixes text in `input.txt` and writes to `output.txt`. |

The JamSpell HTTP API server allows you to interact with the spell corrector over HTTP. You can start the server by providing a trained language model file, a hostname, and a port number as command-line arguments.

Usage:

./jamspell_server <model.bin> <hostname> <port>

Arguments:

• model.bin: Path to the trained JamSpell language model file.
• hostname: The network interface to bind to (e.g., localhost or 0.0.0.0).
• port: The port number to listen on (e.g., 8080).

# Example command to start the server
./jamspell_server model.bin localhost 8080