Piper Text-to-Speech Engine

To use the Piper HTTP Server, you must install the necessary dependencies into your Python virtual environment using the provided requirements file.

.venv/bin/pip3 install -r requirements_http.txt

Once training is complete, export the PyTorch checkpoint (.ckpt) to ONNX format. You must also copy the config.json from your training directory to the model location, renamed to match the ONNX file.

Note: The export script can be further optimized using onnx-simplifier.

# Export to ONNX
python3 -m piper_train.export_onnx \
    /path/to/model.ckpt \
    /path/to/model.onnx

# Copy config to required location
cp /path/to/training_dir/config.json /path/to/model.onnx.json

# Test the exported model
echo 'This is a test.' | \
  piper -m /path/to/model.onnx --output_file test.wav

Start the web server by running the piper.http_server module via Python. You must specify a model using the --model flag. Use --help to view all available command-line options.

.venv/bin/python3 -m piper.http_server --model ...

To set up a training environment, install the system dependencies and create a Python virtual environment. You must also build the monotonic alignment extension and ensure espeak-ng is installed.

1. Install python3-dev and espeak-ng via apt.
2. Create and activate a Python virtual environment in piper/src/python.
3. Install the package in editable mode.
4. Run the build_monotonic_align.sh script located in src/python to build the necessary extension.

sudo apt-get install python3-dev espeak-ng

cd piper/src/python
python3 -m venv .venv
source .venv/bin/activate
pip3 install --upgrade pip
pip3 install --upgrade wheel setuptools
pip3 install -e .

# Run the build script
./build_monotonic_align.sh

To generate audio from checkpoints during training, use python3 -m piper_train.infer.

Input format for inference is a .jsonl file where each line is a JSON object containing phoneme_ids (and speaker_id for multi-speaker models). You can generate these test files using piper-phonemize.

# Generate audio from test sentences
cat test_en-us.jsonl | \
    python3 -m piper_train.infer \
        --sample-rate 22050 \
        --checkpoint /path/to/training_dir/lightning_logs/version_0/checkpoints/*.ckpt \
        --output-dir /path/to/training_dir/output

# Generate phoneme JSONL from text using piper-phonemize
lib/piper_phonemize -l en-us --espeak-data lib/espeak-ng-data/ < my_test_sentences.txt > my_test_phonemes.jsonl

Piper training requires a config.json and a dataset.jsonl file. These are generated using the piper_train.preprocess module.

Input Dataset Format

The pre-processing script expects a directory containing:

• wav/: A directory containing audio files.
• metadata.csv: A pipe-delimited (|) CSV file with no header.

Single-speaker format: id|text (where id is the filename without extension).

Multi-speaker format: id|speaker|text (where speaker is the name of the speaker).

Pre-processing Command

Use the following command to generate the training files. For multi-speaker datasets, omit the --single-speaker flag.

# Single-speaker example
python3 -m piper_train.preprocess \
  --language en-us \
  --input-dir /path/to/dataset_dir/ \
  --output-dir /path/to/training_dir/ \
  --dataset-format ljspeech \
  --single-speaker \
  --sample-rate 22050

The preprocessing script produces two main files in the --output-dir:

1. config.json: Contains metadata for training, including dataset name, audio sample rate and quality, espeak voice settings, phoneme_type, num_symbols, and speaker_id_map.
2. dataset.jsonl: A JSON Lines file where each line represents an Utterance. Each entry contains:
   • text: The original text.
   • audio_path: Path to the original audio.
   • speaker: Speaker name (if multi-speaker).
   • speaker_id: Integer ID for the speaker.
   • phonemes: List of phoneme strings.
   • phoneme_ids: List of integer IDs corresponding to phonemes.
   • audio_norm_path: Path to the normalized audio file.
   • audio_spec_path: Path to the audio spectrogram file.

Piper requires audio files to be in a specific format for successful training:

• Format: .wav
• Sample Rate: 16000 or 22050 Hz
• Bit Depth: 16-bit
• Channels: mono
• Naming: It is recommended to use sequential numbering (e.g., 1.wav, 2.wav).

If your audio files do not match the target sample rate, you can use the resample.py utility provided in the source to convert them.

python resample.py --input_dir "/content/dataset/wavs" --output_dir "/content/dataset/wavs_resampled" --output_sr 22050 --file_ext "wav"

Piper requires audio files and transcription metadata to be formatted specifically:

Audio Requirements

• Format: .wav
• Specifications: 16000 or 22050Hz, 16-bit, mono.
• Naming: It is recommended to use numbered files (e.g., 1.wav, 2.wav).

Transcription Metadata (metadata.csv)

The transcription file must be a UTF-8 (without BOM) .csv file. The structure depends on whether you are training a single or multiple speakers:

Single Speaker Format: wavs/1.wav|This is the text spoken in audio 1.

Multi-Speaker Format: wavs/speaker1audio1.wav|speaker1|This is the text spoken by speaker 1.

Piper uses two primary methods for phonemization, determined by the phoneme_type key in the config.json:

1. ESpeak (espeak): Uses the espeak engine. If the voice is set to Arabic (ar), it applies Arabic diacritization using tashkeel_run before phonemizing.
2. Text (text): Uses phonemize_codepoints to map text directly to codepoints.

The phonemize function returns a list of lists, where each inner list represents a group of phonemes for a single sentence.

Special Tokens:

• BOS (^): Beginning of sentence.
• EOS ($): End of sentence.
• PAD (_): Padding token.