DOCSEARCHbyAlgolia
Submit Docs

translateLocally

repository·master·Indexed 20 days ago

https://github.com/xapajiamnu/translatelocally

A fast and secure local machine translation tool with a GUI, powered by marian and Bergamot. It supports command-line usage for text translation and model management, as well as integration via Native Messaging for browser extensions (Firefox/Chrome). The tool allows for importing custom Marian models, 8-bit integer quantization, and the use of student models for optimized performance.

Tokens
11K
Snippets
37
Records
43
Agent score
68%

What's inside translateLocally

  1. Integrate with Native Messaging (Python/Browser)

    master

    The translateLocally NativeMessaging interface allows integration with browser extensions (Firefox/Chrome) or other applications using JSON-formatted messages over stdin/stdout.

    Key Constraints:

    • Incoming message limit: 10MB (matches Firefox limits).
    • Response limit: ~4GB.

    Using with Python: Start translateLocally as a subprocess using the -p option. Pass JSON messages to its stdin. For a complete implementation, refer to the scripts/native_client.py example which demonstrates an async Python API.

    Using with Browser Extensions:

    • Firefox: translateLocally automatically registers with Firefox when run in GUI mode. Install the Firefox translation addon and select "translateLocally" as the provider.
    • Chrome: Functionality is automatically available.
    • Custom Extensions: To develop your own, you must add your extension ID to src/constants.h in the source code and rebuild translateLocally before it will accept messages from your extension.
    # Start in subprocess mode for Native Messaging
./translateLocally -p

  2. Quantise a model to 8-bit integers

    master

    To create an efficient model for translateLocally, you should quantise the model to 8-bit integers using the Bergamot fork of marian. Use the marian-conv tool to generate the quantised .bin file from an .npz input model.

    After quantisation, you must update your configuration file (e.g., config.intgemm8bitalpha.yml) to:

    1. Point to the new .bin model file.
    2. Append gemm-precision: int8shift to the configuration.
    $MARIAN/marian-conv -f input_model.npz -t output_model.bin --gemm-type intgemm8

  3. Install build dependencies for Ubuntu

    master

    Depending on your Ubuntu version, install the following packages to prepare for compilation:

    Ubuntu 20.04: Requires libpcre++-dev, qttools5-dev, qtbase5-dev, libqt5svg5-dev, libarchive-dev, and libpcre2-dev.

    Ubuntu 22.04: Requires libxkbcommon-x11-dev, libpcre++-dev, libvulkan-dev, libgl1-mesa-dev, qt6-base-dev, qt6-base-dev-tools, qt6-tools-dev, qt6-tools-dev-tools, qt6-l10n-tools, qt6-translations-l10n, libqt6svg6-dev, libarchive-dev, and libpcre2-dev.

    Installing Intel MKL (Required for Ubuntu): To install the required intel-mkl-64bit-2020.0-088:

    wget -qO- "https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB" | sudo apt-key add -
sudo sh -c "echo deb https://apt.repos.intel.com/mkl all main > /etc/apt/sources.list.d/intel-mkl.list"
sudo apt-get update -o Dir::Etc::sourcelist="/etc/apt/sources.list.d/intel-mkl.list"
sudo apt-get install -y --no-install-recommends intel-mkl-64bit-2020.0-088
    # Ubuntu 20.04
sudo apt-get install -y libpcre++-dev qttools5-dev qtbase5-dev libqt5svg5-dev libarchive-dev libpcre2-dev

# Ubuntu 22.04
sudo apt-get install -y libxkbcommon-x11-dev libpcre++-dev libvulkan-dev libgl1-mesa-dev qt6-base-dev qt6-base-dev-tools qt6-tools-dev qt6-tools-dev-tools qt6-l10n-tools qt6-translations-l10n libqt6svg6-dev libarchive-dev libpcre2-dev

  4. Compile translateLocally from source

    master

    To build translateLocally from source, use cmake and make.

    Standard Build Process:

    1. Create a build directory.
    2. Run cmake ...
    3. Run make -j5.
    4. Execute ./translateLocally.

    Note for ARM architectures: If compiling on ARM, you may need to run a specific fix script provided by the build environment. The exact command path is typically provided by cmake during the configuration step (e.g., $GITHUB_WORKSPACE/cmake/fix_ruy_build.sh ...).

    Dependencies: Requires QT>=5, libarchive, and intel-mkl-static. QT6 is fully supported and encouraged. Depending on your distribution, you may need specific packages like qt6-base-dev, libqt6svg6-dev, or qt6-tools-dev.

    mkdir build
cd build
cmake ..
# (Optional ARM fix step if required by cmake output)
make -j5
./translateLocally

  5. Optimize translation performance with student models

    master

    For maximum performance, use student models instead of teacher models (like transformer-base). Student models are typically at least 8X faster.

    To achieve even higher performance (an additional 30%-40% boost), you can:

    1. Precompute the quantisation multipliers of the model.
    2. Use a lexical shortlist.

    Note: These optimizations require the Bergamot fork of Marian. Detailed instructions for creating student models and performing 8-bit quantization can be found in the Bergamot project documentation.

  6. Import custom Marian models

    master

    You can import custom models trained with marian. The model must be packaged as a .tar.gz archive containing a specific directory structure and metadata.

    Required Directory Structure:

    my-custom-model/
├── config.intgemm8bitalpha.yml
├── model_info.json
├── model.npz
└── vocab.deen.spm
    • config.intgemm8bitalpha.yml: Must use this exact name. Contains standard marian configuration options.
    • model_info.json: Must use this exact name. Contains metadata (e.g., modelName, shortName, type, src, trg, version).
    • Other files (like model.npz or vocab files) can have arbitrary names.

    Import Steps:

    1. Organize files into the structure above.
    2. Create an archive: tar -czvf my-custom-model.tar.gz my-custom-model.
    3. In the translateLocally GUI, go to Edit -> Translator Settings -> Languages -> Import model and select your archive.
    # Create the archive for import
tar -czvf my-custom-model.tar.gz my-custom-model

  7. Build and package for MacOS

    master

    On MacOS, translateLocally uses Apple Accelerate instead of MKL.

    To produce a .dmg: It is recommended to use the Qt distribution of Qt rather than Homebrew to avoid issues with macdeployqt.

    mkdir build
cd build
cmake ..
cmake --build . -j3 --target translateLocally-bin translateLocally.dmg

    To sign and notarize a .dmg: You can use the provided dist/macdmg.sh script. Ensure you set the necessary environment variables for signing/notarization before running.

    mkdir build
cd build
cmake ..
make -j5
../dist/macdmg.sh .

  8. Add custom model repositories

    master

    You can extend the available models by adding custom repositories via the Settings -> Repositories menu in the application.

    Currently supported repositories include:

    • Bergamot (Default): https://translatelocally.com/models.json
    • OpusMT: https://object.pouta.csc.fi/OPUS-MT-models/app/models.json (Must be added manually)
    • HPLT: https://raw.githubusercontent.com/hplt-project/bitextor-mt-models/refs/heads/main/models.json

  9. Manage models via CLI

    master

    You can manage translation models using the following CLI workflows:

    1. List available models (online):

    ./translateLocally -a

    This returns a list of models with their type and version. Use the provided identifier with the -d flag to download.

    2. Download a model:

    ./translateLocally -d <model-identifier>

    Example: ./translateLocally -d en-et-tiny

    3. List locally installed models:

    ./translateLocally -l

    4. Remove a model:

    ./translateLocally -r <model-identifier>
    # List available online
./translateLocally -a

# Download a specific model
./translateLocally -d en-et-tiny

# List local models
./translateLocally -l

# Remove a model
./translateLocally -r en-et-tiny

  10. Build with OpenBLAS (Not Recommended)

    master

    While translateLocally supports building against OpenBLAS, it is strongly discouraged due to significant performance penalties (up to 100x slowdown) caused by OpenMP parallelization on small matrices.

    If you must use OpenBLAS, you must disable OpenMP parallelization at runtime using OMP_NUM_THREADS=1.

    Build Command:

    mkdir build
cd build
cmake .. -DBLAS_LIBRARIES=-lblas -DCBLAS_LIBRARIES=-lcblas
make -j6
OMP_NUM_THREADS=1 ./translateLocally

  11. Native Messaging Protocol Overview

    master

    The translateLocally CLI uses a JSON-based native messaging protocol for communication. All requests and responses follow a structured format using a unique id to track asynchronous operations.

    Request Format

    {
  "id": int,
  "command": "string",
  "data": { ... command specific fields }
}

    Success Response Format

    {
  "id": int,
  "success": true,
  "data": { ... command specific fields }
}

    Error Response Format

    {
  "id": int,
  "success": false,
  "error": "string error message"
}

    Update Format (for long-running tasks like downloads)

    {
  "id": int,
  "update": true,
  "data": { ... command specific fields }
}

  12. Manage application settings with the Settings class

    master

    The Settings class acts as a central container for all application configuration. It manages a QSettings instance and provides individual settings via SettingImpl<T> objects. Each setting can be read or written, and they support a signal-based mechanism to notify other parts of the application when a value changes.

    Key Concepts

    Setting Behavior

    When updating a setting using setValue, you can specify a Behavior to control when the valueChanged signal is emitted:

    • AlwaysEmit: The signal is emitted every time setValue is called.
    • EmitWhenChanged: The signal is only emitted if the new value is different from the current value (default behavior).

    Accessing Values

    Values can be accessed using the .value() method or by using the function call operator () as a shorthand.

    // Example of accessing and updating settings
Settings settings;

// Read a value
bool immediate = settings.translateImmediately.value();

// Update a value (emits valueChanged by default)
settings.cores.setValue(4);

// Update a value and force signal emission
settings.cores.setValue(4, Settings::Setting::AlwaysEmit);

// Connect to changes
QObject::connect(&settings.cores, &Setting::valueChanged, [](QString name, QVariant value) {
    qDebug() << "Setting" << name << "changed to" << value;
});