DOCSEARCHbyAlgolia
Submit Docs

MegaTTS 3 Documentation

repository·main·Indexed 27 days ago

https://github.com/bytedance/megatts3

A PyTorch implementation of a lightweight and efficient Text-to-Speech (TTS) system featuring high-quality voice cloning, bilingual support for Chinese and English, and controllable accent and pronunciation parameters. Includes submodules such as an Aligner, a Graphme-to-Phoneme model based on Qwen2.5-0.5B, and WaveVAE for waveform compression and vocoding.

Tokens
1.6K
Snippets
4
Records
7
Agent score
41%

What's inside MegaTTS 3

  1. Download MegaTTS 3 Pretrained Checkpoints

    main

    Download pretrained checkpoints from Google Drive or Huggingface and place them in the ./checkpoints/xxx directory.

    Important Security Note: WaveVAE encoder parameters are not uploaded. For inference with specific speakers, you must use pre-extracted .npy voice latents. If you want to synthesize speech for speaker A, you need both A.wav and A.npy in the same directory.

  2. Install MegaTTS 3 on Linux

    main

    To set up MegaTTS 3 on a Linux environment, clone the repository, create a Python 3.10 conda environment, and install the required dependencies. You must also set the PYTHONPATH to the root directory of the project.

    Note: If you encounter pydantic bugs during inference, ensure pydantic and gradio versions are matched. If you encounter httpx bugs, check if your no_proxy environment variable contains :: patterns.

    # Clone the repository
git clone https://github.com/bytedance/MegaTTS3
cd MegaTTS3

# Create a python 3.10 conda env
conda create -n megatts3-env python=3.10
conda activate megatts3-env
pip install -r requirements.txt

# Set the root directory
export PYTHONPATH="/path/to/MegaTTS3:$PYTHONPATH"

# [Optional] Set GPU
export CUDA_VISIBLE_DEVICES=0

  3. Install MegaTTS 3 using Docker

    main

    You can run MegaTTS 3 using Docker. Ensure you have downloaded the pretrained checkpoints before building/running. The Docker version is currently under testing.

    To run with GPU support (local-only), use the --gpus all flag. To expose the service remotely, you must set GRADIO_SERVER_NAME=0.0.0.0 and provide GRADIO_USERNAME and GRADIO_PASSWORD.

    # Build the image
docker build . -t megatts3:latest

# For GPU inference (local-only)
docker run -it -p 127.0.0.1:7860:7860 --gpus all -e CUDA_VISIBLE_DEVICES=0 megatts3:latest

# For CPU inference (local-only)
docker run -it -p 127.0.0.1:7860:7860 megatts3:latest

# Remote exposure with auth
docker run -it -p 7860:7860 --gpus all \
  -e CUDA_VISIBLE_DEVICES=0 \
  -e GRADIO_SERVER_NAME=0.0.0.0 \
  -e GRADIO_USERNAME=admin \
  -e GRADIO_PASSWORD=change_me \
  megatts3:latest

  4. Install MegaTTS 3 on Windows

    main

    Windows support is currently under testing. To install, create a Python 3.10 conda environment and install dependencies. Note that WeTextProcessing==1.0.4.1 should be commented out in requirements.txt and replaced with version 1.0.3. You may also need to install pynini via conda-forge and ffmpeg via conda if you encounter ffprobe or ffmpeg errors.

    # Comment below dependence in requirements.txt:
# WeTextProcessing==1.0.4.1

# Create a python 3.10 conda env
conda create -n megatts3-env python=3.10
conda activate megatts3-env
pip install -r requirements.txt
conda install -y -c conda-forge pynini==2.1.5
pip install WeTextProcessing==1.0.3

# [Optional] For GPU inference
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

# Set environment variable for root directory
set PYTHONPATH="C:\path\to\MegaTTS3;%PYTHONPATH%" # Windows
$env:PYTHONPATH="C:\path\to\MegaTTS3;%PYTHONPATH%" # Powershell on Windows

# [Optional] Set GPU
set CUDA_VISIBLE_DEVICES=0 # Windows
$env:CUDA_VISIBLE_DEVICES=0 # Powershell on Windows

  5. Run MegaTTS 3 Inference via CLI

    main

    Use tts/infer_cli.py for command-line text-to-speech.

    Arguments:

    • --input_wav: Path to the prompt audio file.
    • --input_text: The text to synthesize.
    • --output_dir: Directory to save generated audio.
    • --p_w (intelligibility weight): Controls how much the output follows standard pronunciation. Lower values (e.g., 1.0) retain more of the speaker's original accent.
    • --t_w (similarity weight): Controls expressiveness and similarity. Increasing this (range 2.0~5.0) can improve emotional cases. It is typically set 0–3 points higher than p_w.

  6. Run MegaTTS 3 via Web UI (Gradio)

    main

    Launch a Gradio-based Web UI for inference using tts/gradio_api.py. By default, it is local-only. To expose it remotely, set the GRADIO_SERVER_NAME, GRADIO_USERNAME, and GRADIO_PASSWORD environment variables.

    # Local-only
python tts/gradio_api.py

# Remote exposure with auth
GRADIO_SERVER_NAME=0.0.0.0 GRADIO_USERNAME=admin GRADIO_PASSWORD=change_me python tts/gradio_api.py

  7. MegaTTS 3 Submodules Reference

    main

    MegaTTS 3 includes several specialized submodules:

    • Aligner: A robust speech-text aligner used for preparing finetuning datasets, filtering noisy speech datasets, phoneme recognition, and speech segmentation.
    • Graphme-to-Phoneme Model: A Qwen2.5-0.5B model finetuned for robust graphme-to-phoneme conversion.
    • WaveVAE: A waveform VAE that compresses 24 kHz speech into 25 Hz acoustic latents. It can be used for compact training targets, voice conversion, or as a high-quality vocoder.