DOCSEARCHbyAlgolia
Submit Docs

MeloTTS Documentation

repository·main·Indexed 27 days ago

https://github.com/myshell-ai/melotts

A high-quality, multi-lingual text-to-speech library developed by MIT and MyShell.ai. MeloTTS supports real-time inference on CPUs and features multi-accent English and mixed-language Chinese support. Supported languages include English, Spanish, French, Chinese, Japanese, and Korean. The library provides a Python API, a CLI, a WebUI, and tools for training on custom datasets.

Tokens
2.5K
Snippets
11
Records
20
Agent score
93%

What's inside MeloTTS

  1. Overview of MeloTTS features and supported languages

    main

    MeloTTS is a high-quality multi-lingual text-to-speech library designed for fast, CPU real-time inference.

    Supported languages include:

    • English (American, British, Indian, Australian, and Default)
    • Spanish
    • French
    • Chinese (supports mixed Chinese and English)
    • Japanese
    • Korean

  2. Access TTS models on MyShell

    main

    You can access a wide variety of Text-to-Speech (TTS) models and voices through MyShell widgets. These include various languages and voice profiles such as:

    • English: British male, cheerful female, sultry male.
    • Spanish: Female, young male, girl.
    • French: Girl, soft male, soft female.
    • German: Soft male, soft female, girl.
    • Portuguese: Clear female, boy, sober male.
    • Russian: Mature female, mature male.
    • Chinese: Sweet female, young male.

    Explore more models at the MyShell Widget Center.

  3. Get started with MeloTTS

    main

    You can use MeloTTS in several ways depending on your needs:

    1. Use without Installation: Follow the guide in docs/quick_use.md for immediate access.
    2. Install and Use Locally: Follow the guide in docs/install.md for local setup.
    3. Training on Custom Dataset: Follow the guide in docs/training.md to fine-tune the model.

  6. Prepare training data for MeloTTS

    main

    Training requires audio files (recommended 44100Hz) and a metadata file. The metadata file must follow a pipe-delimited format: path/to/audio.wav |<speaker_name>|<language_code>|<text>.

    Example format:

    path/to/audio_001.wav |<speaker_name>|<language_code>|<text_001>
path/to/audio_002.wav |<speaker_name>|<language_code>|<text_002>

  7. Install MeloTTS on Linux and macOS

    main

    To install MeloTTS locally on Linux (tested on Ubuntu 20.04) or macOS, clone the repository, install the package in editable mode, and download the required unidic data.

    git clone https://github.com/myshell-ai/MeloTTS.git
cd MeloTTS
pip install -e .
python -m unidic download
    git clone https://github.com/myshell-ai/MeloTTS.git
cd MeloTTS
pip install -e .
python -m unidic download

  8. Preprocess training data

    main

    Run the preprocessing script using your metadata file. This will generate a configuration file (e.g., config.json) which can be manually edited to adjust hyperparameters, such as decreasing the batch size to avoid CUDA out-of-memory errors.

    python preprocess_text.py --metadata data/example/metadata.list

  10. Install MeloTTS using Docker

    main

    For Windows or macOS users experiencing compatibility issues, use Docker.

    Build the image:

    git clone https://github.com/myshell-ai/MeloTTS.git
cd MeloTTS
docker build -t melotts .

    Run the container: To run without GPU:

    docker run -it -p 8888:8888 melotts

    To run with GPU support:

    docker run --gpus all -it -p 8888:8888 melotts

    Once running, access the WebUI at http://localhost:8888.

    git clone https://github.com/myshell-ai/MeloTTS.git
cd MeloTTS
docker build -t melotts .
docker run --gpus all -it -p 8888:8888 melotts

  12. Process and phonemize Spanish sentences

    main

    To process Spanish text, you can implement a pipeline that cleans whitespace, handles punctuation (specifically ¿, —, and ¡), splits the text into sentences, and then applies es2ipa to each sentence.

    Note: The implementation requires adding the path containing es_to_ipa to sys.path if it is not installed as a package.

    import re
import sys
from es_to_ipa import es2ipa

def split_sentences_en(text, min_len=10):
    # Replace newlines, tabs, and spaces with a single space
    text = re.sub('[
	 ]+', ' ', text)
    # Add a delimiter after specific punctuation
    text = re.sub('([¿—¡])', r'\1 $#!', text)
    
    # Split sentences and strip whitespace
    sentences = [s.strip() for s in text.split(' $#!')]
    if len(sentences[-1]) == 0: 
        del sentences[-1]

    new_sent = []
    for ind, sent in enumerate(sentences):
        if sent in ['¿', '—', '¡']:
            new_sent.append(sent)
        else:
            new_sent.append(es2ipa(sent))

    return ''.join(new_sent)

# Usage
text = '—¿Habéis estado casada alguna vez?'
print(split_sentences_en(text))