DOCSEARCHbyAlgolia
Submit Docs

Kitten TTS

repository·main·Indexed 12 days ago

https://github.com/kittenml/kittentts

An open-source, ultra-lightweight text-to-speech library built on ONNX with just 15 million parameters. Designed for high-quality voice synthesis on CPUs and edge deployment, it supports streaming audio, text normalization, and GPU acceleration via CUDA or ROCm. Version 0.8.1 provides methods for synthesizing speech to NumPy arrays or files using a variety of built-in voices.

Tokens
7K
Snippets
33
Records
34
Agent score
95%

What's inside Kitten TTS

  1. Install Kitten TTS

    main

    Install the Kitten TTS library using pip from the official release wheel.

    Prerequisites:

    • Python 3.8 or later
    • pip

    Note: It is recommended to use a virtual environment (conda, venv, etc.) to avoid dependency conflicts.

    pip install https://github.com/KittenML/KittenTTS/releases/download/0.8.1/kittentts-0.8.1-py3-none-any.whl

  2. Use GPU acceleration with Kitten TTS

    main

    While optimized for CPU, you can use a GPU by installing the GPU requirements and specifying the cuda backend during model initialization.

    1. Install GPU requirements:
    pip install -r requirements_gpu.txt
    1. Initialize with backend="cuda":
    from kittentts import KittenTTS

m = KittenTTS("KittenML/kitten-tts-mini-0.8", backend="cuda")

  3. Synthesize speech directly to a file with model.generate_to_file()

    main

    The generate_to_file method synthesizes speech and writes the result directly to an audio file.

    Parameters:

    • text (str): Input text.
    • output_path (str): Path to save the audio file.
    • voice (str): Voice name.
    • speed (float, default: 1.0): Speech speed multiplier.
    • sample_rate (int, default: 24000): Audio sample rate in Hz.
    • clean_text (bool, default: True): Preprocess text (expands numbers, currencies, etc.).
    from kittentts import KittenTTS

model = KittenTTS("KittenML/kitten-tts-mini-0.8")
model.generate_to_file("Hello, world.", "output.wav", voice="Bruno", speed=0.9)

  4. Normalize text with normalize_text()

    main

    The normalize_text function converts text into a format suitable for TTS (e.g., expanding abbreviations, dates, and currencies) without generating audio.

    Parameters:

    • text (str): Input text.
    • locale (str, default: "en-US"): Locale for normalization.
    • return_spans (bool, default: False): If True, returns an object containing the normalized text and character spans mapping the original text to the normalized segments.
    from kittentts import normalize_text

# Basic normalization
normalized = normalize_text("Dr. Rivera paid $12.50 at 3:05 p.m.")
# Result: "Doctor Rivera paid twelve dollars and fifty cents at three oh five p m."

# Normalization with spans
result = normalize_text("Fig. 2", return_spans=True)
print(result.text)
print(result.spans)

  5. Synthesize speech with model.generate()

    main

    The generate method synthesizes speech from text and returns a NumPy array of audio samples at 24 kHz.

    Parameters:

    • text (str): Input text to synthesize.
    • voice (str): Voice name. Available voices: ['Bella', 'Jasper', 'Luna', 'Bruno', 'Rosie', 'Hugo', 'Kiki', 'Leo'].
    • speed (float, default: 1.0): Speech speed multiplier.
    • clean_text (bool, default: False): If True, preprocesses text (expands numbers, currencies, etc.).
    from kittentts import KittenTTS
import soundfile as sf

model = KittenTTS("KittenML/kitten-tts-mini-0.8")
# Generate audio as a NumPy array
audio = model.generate("This high-quality TTS model runs without a GPU.", voice="Jasper")

# Save to file using soundfile
sf.write("output.wav", audio, 24000)

  6. Initialize the KittenTTS model

    main

    Use the KittenTTS class to load a model from the Hugging Face Hub. You can specify a model_name (the Hugging Face repository ID) and an optional cache_dir for local storage.

    from kittentts import KittenTTS

# Load a specific model
model = KittenTTS("KittenML/kitten-tts-mini-0.8")

  8. Reference: Available Models

    main

    Kitten TTS provides several model variants optimized for different parameter counts and sizes. Note that kitten-tts-nano-0.8-int8 may have reported issues.

    | Model | Parameters | Size |
|---|---|---| 
| kitten-tts-mini | 80M | 80 MB |
| kitten-tts-micro | 40M | 41 MB |
| kitten-tts-nano | 15M | 56 MB |
| kitten-tts-nano (int8) | 15M | 25 MB |

  9. Normalize text using KittenTTS.normalize_text()

    main

    The normalize_text method prepares text for synthesis without generating audio. This is useful for preprocessing or inspecting how text will be interpreted.

    Arguments

    • text (str): Input text.
    • locale (str): The locale for normalization. Defaults to "en-US".
    • return_spans (bool): If True, returns spans along with the normalized text. Defaults to False.
    normalized = tts.normalize_text("Hello, world!", locale="en-US")

  10. Initialize KittenTTS with the KittenTTS class

    main

    The KittenTTS class is the primary interface for text-to-speech synthesis. When initializing, you can provide a Hugging Face repository ID or a short model name. If a short name is provided (without a /), it is automatically prefixed with KittenML/.

    Arguments

    • model_name (str): Hugging Face repository ID or model name. Defaults to "KittenML/kitten-tts-nano-0.8".
    • cache_dir (str, optional): Directory to cache downloaded model files.
    • backend (optional): The backend to use for inference.
    from kittentts import KittenTTS

# Initialize with a specific model
tts = KittenTTS(model_name="KittenML/kitten-tts-nano-0.8", cache_dir="./models")

  11. Initialize KittenTTS_1_Onnx

    main

    Use the KittenTTS_1_Onnx class to interface with the ONNX model for text-to-speech synthesis. You must provide the path to the .onnx model file and a .npz file containing voice data.

    Arguments:

    • model_path (str): Path to the ONNX model file. Defaults to "kitten_tts_nano_preview.onnx".
    • voices_path (str): Path to the voices NPZ file.
    • speed_priors (dict, optional): A dictionary mapping voice names to speed multipliers.
    • voice_aliases (dict, optional): A dictionary to map user-friendly names to internal voice IDs.
    • backend (str, optional): Specifies the execution provider. Supported values are:
      • "cuda": Uses CUDAExecutionProvider.
      • "amd_gpu": Uses ROCMExecutionProvider.
      • "cpu": Uses CPUExecutionProvider.
      • None: Uses default providers.
    from kittentts.onnx_model import KittenTTS_1_Onnx

model = KittenTTS_1_Onnx(
    model_path="path/to/model.onnx",
    voices_path="path/to/voices.npz",
    backend="cuda"
)

  12. Normalize English text for TTS with normalize_text

    main

    The normalize_text function is the primary entry point for preparing English text for TTS. It performs a comprehensive suite of substitutions including URLs, emails, dates, times, currency, percentages, ordinals, abbreviations, and model versions.

    Arguments:

    • text (str): The input text.
    • locale (str): Currently only "en-US" is supported.
    • return_spans (bool): If True, returns a NormalizedTextResult object containing metadata about what was changed.

    Returns:

    • If return_spans=False: A str of the normalized text.
    • If return_spans=True: A NormalizedTextResult object.

    Example of NormalizedTextResult: When return_spans=True, you receive an object with:

    • text: The normalized string.
    • spans: A list of NormalizedSpan objects mapping original character indices to normalized indices, including the reason for the change (e.g., "url", "date", "currency").
    from kittentts.preprocess import normalize_text

# Simple usage
text = "Contact me at test@example.com or visit https://google.com"
print(normalize_text(text))
# "Contact me at test at example dot com or visit www dot google dot com"

# Usage with span metadata
result = normalize_text(text, return_spans=True)
for span in result.spans:
    print(f"Reason: {span.reason}, Original: {span.originalStartChar}-{span.originalEndChar}")