DOCSEARCHbyAlgolia
Submit Docs

RealtimeTTS

repository·master·Indexed 26 days ago

https://github.com/koljab/realtimetts

A Python library for low-latency text-to-speech applications capable of converting strings, generators, and LLM token streams into audio. It supports local playback, file output, and streaming via FastAPI HTTP and WebSocket endpoints. Compatible engines include SystemEngine, GTTSEngine, EdgeEngine, OpenAIEngine, AzureEngine, ElevenlabsEngine, CoquiEngine, PiperEngine, and KokoroEngine. Features include Dockerized deployments for OpenAI TTS and ZipVoice, and an XTTSRVCSynthesizer class for Coqui XTTS with RVC post-processing.

Tokens
33.4K
Snippets
109
Records
204
Agent score
86%

What's inside RealtimeTTS

  1. Overview of RealtimeTTS

    master

    RealtimeTTS is a low-latency text-to-speech (TTS) library optimized for real-time applications, such as streaming text from Large Language Models (LLMs). It converts text streams into high-quality audio with minimal delay.

    Key Features:

    • Low Latency: Designed for near-instantaneous conversion, ideal for LLM outputs.
    • High-Quality Audio: Produces natural-sounding speech.
    • Multiple Engine Support: Integrates with OpenAI TTS, Elevenlabs, Azure Speech Services, Coqui TTS, gTTS, and System TTS.
    • Multilingual Support: Capable of handling multiple languages.
    • Fallback Mechanism: Provides reliability by switching to alternative engines if a primary engine fails.

  2. Get started with RealtimeTTS

    master

    RealtimeTTS is a Python library designed to stream text into speech with low latency. It supports plain strings, generators, and LLM token streams. Audio can be played locally, saved to WAV files, or passed as chunks to other processes.

    To begin using the library, follow these steps:

    1. Quick Start: Install a single engine and play your first audio.
    2. Installation: Configure extras, platform-specific requirements, and external tools.
    3. Engine Selection: Choose between local, cloud, voice-cloning, or experimental engines.

  4. Explore RealtimeTTS Examples

    master

    The repository contains several example implementations for different use cases:

    • Local Scripts: tests/simple_test.py provides the smallest possible local script for testing.
    • LLM Integration: tests/simple_llm_test.py demonstrates streaming text from an LLM.
    • Web Servers: The example_fast_api/ directory contains examples for building HTTP and WebSocket-based servers.
    • Voice Cloning & Post-processing: example_rvc/ demonstrates using XTTS combined with RVC (Retrieval-based Voice Conversion) post-processing.
    • Containerization: Docker examples are available in docker/zipvoice/ and docker/openai-tts-docker/.

  5. Install the OmniVoice engine

    master

    To use the OmniVoice engine for local multilingual voice cloning, install the realtimetts package with the omnivoice extra. You must also ensure a compatible PyTorch build is installed for your CUDA or CPU environment, along with the omnivoice package itself.

    pip install "realtimetts[omnivoice]"

# Ensure PyTorch and OmniVoice are also present
pip install torch torchaudio omnivoice

  6. Install the Piper Engine

    master

    To use the Piper engine, install realtimetts with the piper extra. Note that this only installs Python dependencies; you must manually provide the Piper executable and voice model files.

    1. Install the Python package:
      pip install "realtimetts[piper]"
    2. Download or build a Piper executable and a Piper voice model.
    3. Provide the path to the executable either by setting the PIPER_PATH environment variable or by passing piper_path directly to the PiperEngine constructor.
    pip install "realtimetts[piper]"

  7. Install RealtimeTTS

    master

    To perform a fast local smoke test, install the system engine extra via pip:

    pip install "realtimetts[system]"

    OS-Specific Dependencies

    Linux: Install PortAudio headers before installing PyAudio:

    sudo apt-get update
sudo apt-get install python3-dev portaudio19-dev

    macOS:

    brew install portaudio

    For cloud engines, local neural engines, CUDA, or mpv requirements, refer to the detailed installation documentation.

    pip install "realtimetts[system]"

  8. Install the System Engine

    master

    To use the SystemEngine, which utilizes pyttsx3 to wrap operating system TTS voices, install the package with the [system] extra. This installs pyttsx3 and the necessary RealtimeTTS playback dependencies. Note that available voices are determined by your host operating system.

    pip install "realtimetts[system]"

  10. Run ZipVoice via Docker

    master

    A Docker server example is available which exposes a streaming endpoint on port 9086. Use the following commands to build and run the container:

    docker build -t zipvoice-image -f docker/zipvoice/Dockerfile .
docker run --rm --name zipvoice-container -p 9086:9086 --gpus all zipvoice-image
    docker build -t zipvoice-image -f docker/zipvoice/Dockerfile .
docker run --rm --name zipvoice-container -p 9086:9086 --gpus all zipvoice-image

  11. Build and run OpenAI TTS via Docker

    master

    You can run an OpenAI-only TTS engine using Docker. This requires Docker version 24 or higher.

    Steps:

    1. Clone the repository: git clone https://github.com/KoljaB/RealtimeTTS.git
    2. Navigate to the directory: cd RealtimeTTS/docker/openai-tts-docker
    3. Important: Edit the Dockerfile located in RealtimeTTS/docker/openai-tts-docker to set your OPENAI_API_KEY.
    4. Build the image: sudo docker build -t realtime .
    5. Run the container: sudo docker run --privileged -p 8501:8501 realtime
    git clone https://github.com/KoljaB/RealtimeTTS.git
cd RealtimeTTS/docker/openai-tts-docker
# Edit OPENAI_API_KEY in Dockerfile before proceeding
sudo docker build -t realtime .
sudo docker run --privileged -p 8501:8501 realtime