DOCSEARCHbyAlgolia
Submit Docs

EmotiVoice Documentation

repository·main·Indexed 27 days ago

https://github.com/netease-youdao/emotivoice

An open-source, multi-voice, and prompt-controlled text-to-speech (TTS) engine supporting English and Chinese. EmotiVoice features emotional synthesis for controlling speech with emotions such as happy, sad, or angry. The project provides tools for finetuning models using datasets like DataBaker and LJSpeech, an OpenAI-compatible TTS API, a Streamlit interactive demo, and Docker deployment options.

Tokens
4.5K
Snippets
25
Records
35
Agent score
93%

What's inside EmotiVoice

  2. Download and extract LJSpeech data

    main

    Download the LJSpeech-1.1 corpus and extract it into the data/LJspeech/raw directory.

    mkdir data/LJspeech/raw

# download
wget -P data/LJspeech/raw http://data.keithito.com/data/speech/LJSpeech-1.1.tar.bz2
# extract
tar -xjf data/LJspeech/raw/LJSpeech-1.1.tar.bz2 -C data/LJspeech/raw

  3. Full Installation via Conda and Pip

    main

    To install EmotiVoice manually in a local environment, create a Conda environment with Python 3.8 and install the required dependencies:

    conda create -n EmotiVoice python=3.8 -y
conda activate EmotiVoice
pip install torch torchaudio
pip install numpy numba scipy transformers soundfile yacs g2p_en jieba pypinyin pypinyin_dict
python -m nltk.downloader "averaged_perceptron_tagger_eng"

  4. Run EmotiVoice via Docker

    main

    The fastest way to run EmotiVoice is using a Docker image. This requires a machine with an NVIDIA GPU and the NVIDIA container toolkit installed (for Linux or Windows WSL2).

    To run the standard version with a web interface at http://localhost:8501:

    docker run -dp 127.0.0.1:8501:8501 syq163/emoti-voice:latest

    To run a version that includes the OpenAI-compatible TTS API at http://localhost:8000:

    docker run -dp 127.0.0.1:8501:8501 -p 127.0.0.1:8000:8000 syq163/emoti-voice:latest

  6. Prepare Pretrained Model Files

    main

    EmotiVoice requires pretrained models. You can download them using Git LFS from HuggingFace or via ModelScope.

    Option 1: HuggingFace (requires git-lfs)

    git lfs install
git lfs clone https://huggingface.co/WangZeJun/simbert-base-chinese WangZeJun/simbert-base-chinese

    Option 2: ModelScope

    git clone https://www.modelscope.cn/syq163/WangZeJun.git

  7. Install EmotiVoice for training

    main

    To finetune models using the LJSpeech recipe, create a Conda environment with Python 3.8 and install the package with the [train] extra. You must also prepare pre-trained models as specified in the project documentation.

    conda create -n EmotiVoice python=3.8 -y
conda activate EmotiVoice

# Install via pip
pip install EmotiVoice[train]

# Or via git clone
git clone https://github.com/netease-youdao/EmotiVoice
pip install -e .[train]

  8. Configure environment and credentials for EmotiVoice API Demo

    main

    To run the demo, ensure you meet the following requirements:

    1. Python Version: Python 3.6 or higher.
    2. API Credentials: You must obtain an Application ID (应用ID) and an Application Secret (应用密钥) from the Youdao Zhiyun official website.
    3. Parameter Configuration: Locate the code sections marked with Chinese prompts (中文提示) and fill in the required interface parameters. Specific parameter values can be found in the Youdao Zhiyun documentation.

  9. Install EmotiVoice on Linux

    main

    Follow these steps to set up EmotiVoice in a Linux environment. This requires a GPU and CUDA installation.

    1. Create a Conda environment (Python 3.8 is required):

      conda create -n EmotiVoice python=3.8
conda init
conda activate EmotiVoice

    2. Install git-lfs:

      • Ubuntu:
        sudo apt update
sudo apt install git
sudo apt-get install git-lfs
      • CentOS:
        sudo yum update
sudo yum install git
sudo yum install git-lfs

    3. Clone the repository:

      git lfs install
git lfs clone https://github.com/netease-youdao/EmotiVoice.git

    4. Install dependencies:

      pip install torch torchaudio
pip install numpy numba scipy transformers soundfile yacs g2p_en jieba pypinyin pypinyin_dict
python -m nltk.downloader "averaged_perceptron_tagger_eng"
    conda create -n EmotiVoice python=3.8
conda init
conda activate EmotiVoice

  10. Finetune the EmotiVoice model

    main

    Start the training process using torchrun. Checkpoints are saved in exp/DataBaker/ckpt. You can monitor training with TensorBoard using --logdir=exp/DataBaker.

    torchrun \
--nproc_per_node=1 \
--master_port 8008 \
train_am_vocoder_joint.py \
--config_folder exp/DataBaker/config \
--load_pretrained_model True

  11. Install MFA (Montreal Forced Aligner) environment

    main

    For the recommended alignment step, install the required Conda and Pip dependencies for MFA.

    conda install -c conda-forge kaldi sox librosa biopython praatio tqdm requests colorama pyyaml pynini openfst baumwelch ngram postgresql -y
pip install pgvector hdbscan montreal-forced-aligner