tts-rs: High-level Text-To-Speech Interface for Rust

repository·master·Indexed 41 days ago

https://github.com/ndarilek/tts-rs

A cross-platform Text-To-Speech (TTS) library for Rust (v0.27.0) providing a unified interface for system-level speech engines. Supported backends include WinRT, NVDA and Tolk on Windows, Speech Dispatcher on Linux, AVFoundation on macOS/iOS, Android, and the browser's speechSynthesis API via WebAssembly.

Tokens
5.8K
Snippets
24
Records
36
Agent score
50%

What's inside tts-rs

  1. Overview of TTS-RS supported backends

    master

    TTS-RS is a high-level Text-To-Speech (TTS) interface that supports multiple platform backends:

    • Windows:
      • NVDA (via nvdaControllerClient.dll)
      • Screen readers/SAPI (via Tolk, requires tolk Cargo feature)
      • WinRT
    • Linux: Speech Dispatcher
    • macOS/iOS/tvOS/watchOS/visionOS: AVFoundation (macOS 10.14+)
    • Android
    • WebAssembly
  2. Handle asynchronous engine initialization on Android

    master

    On Android, the TTS engine reports readiness on the Java main thread. To prevent deadlocks (especially in NativeActivity apps where the main thread is held by android-activity), Tts::default() returns immediately after the engine is asked to connect.

    Important behaviors:

    • Queuing: Any speech commands issued before the engine is fully ready are queued and replayed in order once the engine connects.
    • synthesize method: Unlike other methods, synthesize returns audio data and must wait for the engine to be ready. You must call synthesize off your main event-loop thread to avoid blocking the application.
  3. Configure TTS-RS for Android

    master

    TTS-RS is designed to be plug-and-play on Android without requiring manual Java sources, Gradle plugins, or manifest entries, provided the following requirements are met:

    1. minSdkVersion 26 or above: This is required because the UtteranceProgressListener subclass Android callbacks are shipped as an embedded dex and loaded via InMemoryDexClassLoader.
    2. Provide a Context: A Context must be supplied before the first Tts instance is used. You can provide it in two ways:
      • Automatic: The library consults ndk-context by default. If you use android-activity (including via winit or a game engine), it will publish the context there for you.
      • Manual: Call tts::android::set_context with a JavaVM and Context of your choosing. This takes precedence over ndk-context. This is the recommended method if your process outlives its Activity (e.g., a foreground service), as set_context holds its own reference to the application context, whereas ndk-context is typically released when the Activity is destroyed.
  4. Use the Tolk backend on Windows

    master

    The Tolk backend allows for text-to-speech on Windows by leveraging an active screen reader. This backend is only available when a screen reader is running and detected by the system.

    Key Characteristics:

    • Availability: It is only functional if detect_screen_reader() returns a value. Use is_available() to check this state.
    • Supported Features: The backend supports the stop feature. Most other features (like synthesize, pause, resume, or pitch/rate adjustments) are currently unimplemented!() and will result in errors if called.
    • Speech Mechanism: Unlike other backends that might return synthesized audio buffers, the speak method in Tolk sends text directly to the screen reader's speech engine.
  5. Use the Web backend for WASM-based TTS

    master

    The Web backend allows for Text-to-Speech functionality in web environments by leveraging the browser's native speechSynthesis API. It is designed for wasm32 targets and requires a browser environment with a window object.

    Supported Features

    The Web backend supports the following features:

    • stop: Stop all current speech.
    • pause: Pause current speech.
    • rate: Adjust playback speed.
    • pitch: Adjust pitch.
    • volume: Adjust volume.
    • is_speaking: Check if speech is currently active.
    • voice: Get the current voice.
    • get_voice: Retrieve available voices.
    • utterance_callbacks: Receive events like utterance_begin, utterance_end, utterance_pause, utterance_resume, and utterance_stop via the provided Callbacks object.

    Availability Check

    Before attempting to use the backend, you can check if the environment supports speechSynthesis using Web::is_available() (internal/crate-level).

    // Note: Web is a backend implementation for the `Backend` trait.
    // It is intended for use in WASM environments where `web_sys` is available.
  6. Initialize the TTS engine

    master

    To use the library, you can either create a Tts instance with a specific backend or use the default() method to select the most appropriate backend for the current platform.

    Supported backends include:

    • Windows: Nvda, Tolk (requires tolk feature), or WinRt.
    • Linux: Orca or SpeechDispatcher.
    • macOS/iOS/etc: AvFoundation.
    • Android: Android (requires minSdkVersion 26+).
    • WebAssembly: Web.

    Use Tts::backends() to get a list of all available backends on the current platform, ordered by preference.

    use tts::{Tts, Backends};
    
    // Option 1: Use the platform default
    let tts = Tts::default().expect("Failed to initialize default TTS");
    
    // Option 2: Specify a specific backend
    #[cfg(target_os = "linux")]
    let tts = Tts::new(Backends::SpeechDispatcher).expect("Failed to initialize SpeechDispatcher");
  7. Use the NVDA backend for Windows

    master

    The NVDA backend allows tts-rs to interface with the NonVisual Desktop Access (NVDA) screen reader on Windows. This backend relies on the presence of the nvdaControllerClient.dll (or legacy versions like nvdaControllerClient64.dll / nvdaControllerClient32.dll) being shipped with your application or available in the system path.

    To use it, you must ensure NVDA is currently running on the host system. The backend provides speech capabilities, including the ability to stop current speech and check if NVDA is currently speaking (if the installed NVDA version is 2026.3 or newer).

  8. Run the Android hello world example

    master

    To run the Android example, use cargo-apk. You can monitor the output using adb logcat with the tts tag.

    cargo apk run --example android_hello_world
    adb logcat -s tts
    cargo apk run --example android_hello_world
    adb logcat -s tts
  9. Initialize the NVDA backend

    master

    Use Nvda::new() to create a new instance of the NVDA backend. This method attempts to load the required DLLs and verify a connection to the running NVDA instance.

    It returns Err(Error::BackendUnavailable) if:

    1. The controller client DLL fails to load.
    2. NVDA is not currently running.
    let mut backend = Nvda::new().map_err(|e| ...)?;
  10. Speak text using the WinRT backend

    master

    The speak method queues text for playback. If interrupt is set to true and the backend is currently speaking, it will stop the current utterance before starting the new one. This method returns an Option<UtteranceId> which can be used to track the specific utterance.

    // interrupt: true will stop current speech to start new speech
    let utterance_id = backend.speak("Hello, Windows!", true)?;
  11. Configure voice, rate, pitch, and volume on the Web backend

    master

    The Web backend allows fine-grained control over the audio characteristics of the speech.

    Voice Configuration

    • voices(): Returns a Vec<Voice> containing all available voices in the browser.
    • voice(): Returns the currently selected Voice or the browser's default voice.
    • set_voice(&Voice): Sets the active voice. This will return Error::VoiceNotFound if the provided voice ID does not match any available browser voice.

    Audio Parameters

    ParameterMinNormalMax
    Rate0.11.010.0
    Pitch0.01.02.0
    Volume0.01.01.0

    Use set_rate(f32), set_pitch(f32), and set_volume(f32) to update these values.

  12. Initialize the Web backend

    master

    To use the Web backend, initialize it using Web::new(). This requires an Arc<Mutex<Callbacks>> to handle utterance lifecycle events.

    // Assuming callbacks is an Arc<Mutex<Callbacks>>
    let web_backend = Web::new(callbacks)?;
    let web_backend = Web::new(callbacks)?;