Spark-TTS Documentation

You can download the pretrained models using either a Python script via huggingface_hub or via git clone with git-lfs.

# Option 1: Download via python
from huggingface_hub import snapshot_download

snapshot_download("SparkAudio/Spark-TTS-0.5B", local_dir="pretrained_models/Spark-TTS-0.5B")

# Option 2: Download via git clone
mkdir -p pretrained_models

# Make sure you have git-lfs installed
git lfs install

git clone https://huggingface.co/SparkAudio/Spark-TTS-0.5B pretrained_models/Spark-TTS-0.5B

To build the Docker image from scratch and run it manually with GPU support, follow these steps:

1. Build the image using the Dockerfile.server:

docker build . -f Dockerfile.server -t soar97/triton-spark-tts:25.02

2. Run the container with GPU access, host networking, and a shared memory increase:

your_mount_dir=/mnt:/mnt
docker run -it --name "spark-tts-server" --gpus all --net host -v $your_mount_dir --shm-size=2g soar97/triton-spark-tts:25.02

The fastest way to launch the Nvidia Triton Inference Serving for Spark-TTS is by using docker compose.

docker compose up

The run.sh script automates the deployment and testing process using numbered stages.

Command Syntax:

bash run.sh <start_stage> <stop_stage> [service_type]

• <start_stage>: The stage to begin execution from (0-5).
• <stop_stage>: The stage to end execution at (0-5).
• [service_type]: Optional. Specifies 'streaming' or 'offline'. This is required for stages 4 and 5.

Available Stages:

• Stage 0: Download Spark-TTS-0.5B model from HuggingFace.
• Stage 1: Convert HuggingFace checkpoint to TensorRT-LLM format and build TensorRT engines.
• Stage 2: Create the Triton model repository structure and configure model files (adjusts for streaming/offline).
• Stage 3: Launch the Triton Inference Server.
• Stage 4: Run the gRPC benchmark client.
• Stage 5: Run the single utterance client (gRPC for streaming, HTTP for offline).

To install Spark-TTS on Linux, clone the repository and set up a Conda environment with Python 3.12.

Prerequisites:

• Conda (Miniconda recommended)
• Git-LFS (required for model downloading)

Installation Steps:

1. Clone the repository.
2. Create and activate a new Conda environment named sparktts.
3. Install dependencies via requirements.txt.

# Clone the repo
git clone https://github.com/SparkAudio/Spark-TTS.git
cd Spark-TTS

# Create Conda env
conda create -n sparktts -y python=3.12
conda activate sparktts

# Install dependencies
pip install -r requirements.txt

# If you are in mainland China, use the Aliyun mirror:
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com

You can run benchmarks against the running Triton server using stage 4. You must specify the service type (streaming or offline).

Standard Benchmark Commands:

# Run benchmark in streaming mode
bash run.sh 4 4 streaming

# Run benchmark in offline mode
bash run.sh 4 4 offline

Advanced Customization: You can customize parameters like num_task or the dataset directly via client_grpc.py arguments.

Example for streaming with specific tasks and a HuggingFace dataset:

python3 client_grpc.py \
    --server-addr localhost \
    --model-name spark_tts \
    --num-tasks 2 \
    --mode streaming \
    --log-dir ./log_concurrent_tasks_2_streaming_new

Example for customizing the dataset:

python3 client_grpc.py --num-tasks 2 --huggingface-dataset yuekai/seed_tts --split-name wenetspeech4tts --mode [streaming|offline]

Spark-TTS includes a Web UI (built with Gradio) that supports both Voice Cloning (via uploaded or recorded reference audio) and Voice Creation (controlled generation).

python webui.py --device 0

To prepare the models (download, convert to TensorRT-LLM, build engines) and start the Triton Inference Server, run stages 0 through 3.

Note: Stage 2 configures the model repository specifically for either streaming or offline inference. If you switch service types, you must re-run stage 2.

# This runs stages 0, 1, 2, and 3
bash run.sh 0 3

You can perform text-to-speech inference directly from the command line using the cli.inference module. This supports zero-shot voice cloning by providing a prompt audio file and its transcript.

python -m cli.inference \
    --text "text to synthesis." \
    --device 0 \
    --save_dir "path/to/save/audio" \
    --model_dir pretrained_models/Spark-TTS-0.5B \
    --prompt_text "transcript of the prompt audio" \
    --prompt_speech_path "path/to/prompt_audio"

For a quick demonstration of the model, you can run the provided inference script in the example directory.

cd example
bash infer.sh