DOCSEARCHbyAlgolia
Submit Docs

Supertonic TTS

repository·main·Indexed 12 days ago

https://github.com/supertone-inc/supertonic

A high-performance, on-device, multilingual text-to-speech (TTS) system using ONNX Runtime for low-latency, private, and local inference. It supports 31 languages and provides implementations for C++, C# (.NET 9.0+), Go (1.21+), and Flutter (^3.5.0). Key features include automatic text chunking for long-form synthesis, batch inference for multiple voice-text pairs, and adjustable denoising steps to balance audio fidelity and inference speed.

Tokens
16.3K
Snippets
53
Records
81
Agent score
96%

What's inside Supertonic

  1. How long-form and batch inference work

    main

    The Node.js implementation handles text in two distinct modes:

    1. Long-Form Mode (Default): When processing long text without the --batch flag, the system automatically splits text into chunks (default max 300 characters). It synthesizes each chunk and combines them into a single audio file with 0.3s silences between them to ensure natural pacing.

    2. Batch Mode (--batch): Used for processing multiple distinct text/voice pairs at once. In this mode, automatic text chunking is disabled. You must provide a matching number of voice styles and texts (separated by commas and pipes respectively).

    Choose Long-Form mode for single long passages and Batch mode for multiple short, different samples.

  2. How automatic text chunking works for long-form inference

    main

    When not using --batch mode, the system automatically handles long-form text by splitting it into manageable segments based on paragraph and sentence boundaries.

    The process follows these steps:

    1. The long text is split into chunks.
    2. Each chunk is synthesized separately.
    3. Chunks are concatenated into a single audio file with a default 0.3 second pause between segments to ensure natural speech flow.

    This mechanism helps avoid memory issues and maintains natural prosody for long passages.

    Note: This feature is disabled in --batch mode.

  3. Compare Supertonic Model Versions

    main

    Supertonic offers three main model versions depending on your requirements for language support, parameter size, and expression control:

    FeatureSupertonic 3 (Latest)Supertonic 2 (Stable)Supertonic 1 (Legacy)
    Parameters~99M~66M~66M
    Languages3151 (en)
    Expression Tags✅ 10 tags——

    Use Supertonic 3 for the best language coverage and expressive control via tags. Use older versions if you require specific legacy weights or lower parameter counts.

  4. Natural Text Handling and Normalization

    main

    Supertonic is designed to handle complex, real-world text inputs without requiring manual pre-processing or phonetic annotations. It automatically normalizes and correctly pronounces:

    • Financial Expressions: Decimals in currency, abbreviated magnitudes (e.g., M for million, K for thousand), and currency symbols (e.g., $).
    • Phone Numbers: Area codes in parentheses, hyphen separators, and extension notations (e.g., ext. 402).
    • Technical Units: Decimal numbers with units and abbreviated technical notations (e.g., 2.3h for two point three hours, 30kph for thirty kilometers per hour).

  5. Voice Cloning with Supertonic

    main

    The open-weight Supertonic repository focuses on fixed-voice, local TTS and does not include an official voice-cloning pipeline.

    To use custom voices in a local Supertonic deployment:

    1. Use Voice Builder to convert a short reference recording into version-specific JSON files for Supertonic 2 or Supertonic 3.
    2. These JSON files allow your custom voice to be used across supported Supertonic versions.

    For managed workflows (zero-shot cloning, hosted tools, or API integration), use Supertone Play or the Supertone API.

  6. Use Automatic Text Chunking for Long-Form Audio

    main

    For long texts, the system automatically splits the input into chunks based on paragraph and sentence boundaries. Each chunk is synthesized separately and then concatenated with a default 0.3 second pause to ensure natural speech flow and avoid memory issues.

    Note: This feature is active by default when not using the --batch flag. If you use --batch, chunking is disabled.

    mvn exec:java -Dexec.args="--voice-style ../assets/voice_styles/M1.json --text 'Your very long text here...'"
    # Example of Long-Form Inference
mvn exec:java -Dexec.args="--voice-style ../assets/voice_styles/M1.json --text 'This is a very long text that will be automatically split into multiple chunks...'"

  7. Supertonic iOS Project Structure

    main

    The iOS project is a SwiftUI application generated via XcodeGen. The core components are:

    • TTSService.swift: The runtime wrapper containing the TTS inference logic.
    • TTSViewModel.swift: Manages the state and logic for the TTS process.
    • ContentView.swift & App.swift: The SwiftUI user interface and app entry point.
    • AudioPlayer.swift: Handles audio playback.
    • Resources: ONNX models and voice style JSON files located in onnx/ and voice_styles/ directories within the app bundle.

  8. Install Supertonic C++ dependencies

    main

    To use the C++ implementation, you need a C++17 compiler, CMake 3.15+, ONNX Runtime, and nlohmann/json. Installation steps vary by platform:

    Ubuntu/Debian

    sudo apt-get install -y cmake g++ nlohmann-json3-dev
wget https://github.com/microsoft/onnxruntime/releases/download/v1.16.3/onnxruntime-linux-x64-1.16.3.tgz
tar -xzf onnxruntime-linux-x64-1.16.3.tgz
sudo cp -r onnxruntime-linux-x64-1.16.3/include/* /usr/local/include/
sudo cp -r onnxruntime-linux-x64-1.16.3/lib/* /usr/local/lib/
sudo ldconfig

    macOS

    brew install cmake nlohmann-json onnxruntime

    Windows (vcpkg)

    vcpkg install nlohmann-json:x64-windows onnxruntime:x64-windows
vcpkg integrate install

  9. Install Supertonic TTS ONNX Inference Examples (Go)

    main

    To run TTS inference using the Go implementation, follow these steps to set up the environment and dependencies.

    1. Prerequisites

    • Go: Install version 1.21 or later.
    • ONNX Runtime C library:
      • macOS: Install via Homebrew:
        brew install onnxruntime
      • Linux: Download the release from GitHub, extract, and copy the library and include files to /usr/local/lib and /usr/local/include respectively, then run sudo ldconfig.

    2. Install Go Dependencies

    Run the following command in the project directory:

    go mod download

    3. Configure ONNX Runtime Library Path (Optional)

    If the library is not in a standard location, set the ONNXRUNTIME_LIB_PATH environment variable.

    macOS (Automatic Detection):

    export ONNXRUNTIME_LIB_PATH=$(brew --prefix onnxruntime 2>/dev/null)/lib/libonnxruntime.dylib

    Linux (Automatic Detection):

    export ONNXRUNTIME_LIB_PATH=$(find /usr/local/lib /usr/lib -name "libonnxruntime.so*" 2>/dev/null | head -n 1)

    Manual Configuration:

    export ONNXRUNTIME_LIB_PATH=/path/to/libonnxruntime.so  # Linux
export ONNXRUNTIME_LIB_PATH=/path/to/libonnxruntime.dylib  # macOS
    brew install onnxruntime

  10. Synthesize speech with the Python TTS API

    main

    Use the TTS class to generate audio. You can specify a voice style using get_voice_style, set the language (or use "na" for language-agnostic processing), and adjust quality via total_steps and playback speed via speed.

    Key Parameters:

    • lang: Language code (e.g., "en", "ko"). Use "na" to let Supertonic process text language-agnostically.
    • total_steps: Quality setting from 5 (low) to 12 (high). Default is 8.
    • speed: Playback speed from 0.7 (slow) to 2.0 (fast). Default is 1.05.

    Output Formats:

    • wav: A numpy array of shape (1, num_samples,) with dtype=np.float32, sampled at 44100 Hz.
    • duration: A numpy array of shape (1,) containing the duration in seconds.
    from supertonic import TTS

# First run downloads the model from Hugging Face automatically.
tts = TTS(auto_download=True)

style = tts.get_voice_style(voice_name="M1")

text = "Supertonic is a lightning fast, on-device TTS system."

wav, duration = tts.synthesize(
    text=text,
    lang="en",                      # Language code (e.g., "en", "ko", "na" for language-agnostic)
    voice_style=style,              # Voice style object
    total_steps=8,                  # Quality: 5 (low) to 12 (high), default 8 (medium)
    speed=1.05,                     # Speed: 0.7 (slow) to 2.0 (fast),
)

tts.save_audio(wav, "output.wav")

print(f"Generated {duration[0]:.2f}s of audio")

  11. Install and build the Rust TTS ONNX inference examples

    main

    To use the Rust implementation of Supertonic TTS, you must have Rust and Cargo installed. Follow these steps to set up the environment and build the project.

    1. Install Rust

    If you haven't installed Rust yet, use the following command:

    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

    2. Build the project

    Use Cargo to build the project in release mode for optimal performance:

    cargo build --release
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# then
cargo build --release

  12. Use Supertonic in a Flutter application

    main

    To use Supertonic 3 in a Flutter application, this project utilizes the flutter_onnxruntime package.

    Platform Support Note: Currently, only the macOS platform has been verified for this implementation. While flutter_onnxruntime supports other platforms, they have not been tested in this specific example and may require additional verification.

    Multilingual Support: Supertonic 3 supports 31 languages. You can select the appropriate language from the UI dropdown to ensure correct synthesis.

    # Requirements
- Flutter SDK version ^3.5.0

# Running the Demo
```bash
flutter clean
flutter pub get
flutter run -d macos