index-tts-lora

Once the audio tokens and speaker conditions have been extracted and processed, start the fine-tuning process by running the training script.

python train.py

After training is complete, use the inference script to generate synthetic speech using the fine-tuned model.

python indextts/infer.py

Before training, you must extract audio codec tokens and speaker conditions from your dataset.

Input Format: The ${audio_list} file must follow a specific format where each line contains an audio file path and its corresponding transcript, separated by a tab (\t):

/path/to/audio.wav The transcript text goes here.

Command: Run the extract_codec.py tool with the --extract_condition flag.

Output: Processed files and a speaker_info.json file will be generated in the finetune_data/processed_data/ directory. The speaker_info.json contains metadata for each speaker, including avg_duration, sample_num, total_duration_in_seconds, and paths to train_jsonl, valid_jsonl, and medoid_condition.npy files.

# Extract tokens and speaker conditions
python tools/extract_codec.py --audio_list ${audio_list} --extract_condition

After training is complete, use the inference script to synthesize speech using your fine-tuned model.

python indextts/infer.py

The CLI automatically selects the best available hardware if no device is specified:

1. CUDA: If torch.cuda.is_available() is true, it uses cuda:0.
2. MPS: If torch.mps.is_available() is true, it uses mps.
3. CPU: Defaults to cpu and automatically disables --fp16 (as FP16 is not supported for CPU inference in this context).

You can manually override this using the -d or --device flag.

Example: Force CPU usage

python indextts/cli.py "text" -v prompt.wav -d cpu

The speaker_info.json file generated after extraction follows this structure for each speaker entry:

[
    {
        "speaker": "kaishu_30min",
        "avg_duration": 6.6729,
        "sample_num": 270,
        "total_duration_in_seconds": 1801.696,
        "total_duration_in_minutes": 30.028,
        "total_duration_in_hours": 0.500,
        "train_jsonl": "/path/to/kaishu_30min/metadata_train.jsonl",
        "valid_jsonl": "/path/to/kaishu_30min/metadata_valid.jsonl",
        "medoid_condition": "/path/to/kaishu_30min/medoid_condition.npy"
    }
]

The IndexTTS CLI supports the following arguments for controlling output, configuration, and hardware acceleration:

The indextts/cli.py script provides a command-line interface to perform text-to-speech synthesis using a LoRA-based model. You must provide the text to synthesize and a path to a .wav audio prompt file that serves as the voice reference.

Basic Usage:

python indextts/cli.py "Your text here" -v path/to/voice_prompt.wav

Key Requirements:

• Text: A positional argument containing the string you want to synthesize.
• Voice Prompt: A required -v or --voice argument pointing to a .wav file.
• PyTorch: The CLI requires torch to be installed to handle device placement and inference.

python indextts/cli.py "Hello world" -v prompt.wav