Zonos Documentation

System Requirements

• OS: Linux (Ubuntu 22.04/24.04 preferred) or macOS.
• GPU: 6GB+ VRAM. Note that the hybrid model requires an NVIDIA 3000-series GPU or newer.
• CPU: Can run on CPU with sufficient RAM, but performance is significantly slower.

System Dependencies

Zonos requires espeak-ng for phonemization.

• Ubuntu: apt install -y espeak-ng
• macOS: brew install espeak-ng

Python Installation

It is highly recommended to use uv for installation.

Using uv (Recommended)

To install into a new virtual environment:

uv sync
uv sync --extra compile # Required for the hybrid model
uv pip install -e .

To install into an existing environment:

uv pip install -e .
uv pip install -e .[compile] # Required for the hybrid model

Using pip

pip install -e .
pip install --no-build-isolation -e .[compile] # Required for the hybrid model

apt install -y espeak-ng # For Ubuntu
# brew install espeak-ng # For MacOS

You can use Zonos programmatically to perform zero-shot TTS and voice cloning. The workflow involves loading a pretrained model, creating a speaker embedding from a reference audio clip, preparing a conditioning dictionary with text and language, and then generating/decoding the audio.

Available models include:

• Zyphra/Zonos-v0.1-hybrid (Requires 3000-series+ NVIDIA GPU)
• Zyphra/Zonos-v0.1-transformer

import torch
import torchaudio
from zonos.model import Zonos
from zonos.conditioning import make_cond_dict
from zonos.utils import DEFAULT_DEVICE as device

# Load the model
model = Zonos.from_pretrained("Zyphra/Zonos-v0.1-transformer", device=device)

# Prepare speaker embedding from reference audio
wav, sampling_rate = torchaudio.load("assets/exampleaudio.mp3")
speaker = model.make_speaker_embedding(wav, sampling_rate)

# Create conditioning dictionary
cond_dict = make_cond_dict(text="Hello, world!", speaker=speaker, language="en-us")
conditioning = model.prepare_conditioning(cond_dict)

# Generate audio codes
codes = model.generate(conditioning)

# Decode codes to waveform
wavs = model.autoencoder.decode(codes).cpu()
torchaudio.save("sample.wav", wavs[0], model.autoencoder.sampling_rate)

The following conditionings are only applicable for the hybrid model to control speech cleanliness and quality:

• vqscore_8 (FourierConditioner): Encodes desired VQScore (unsupervised speech quality). A value of 0.78 is recommended for high-quality speech. For inference, set all 8 dimensions to the same value. Note: High VQScore correlates with less expressiveness; use 'unconditional' for expressive speech.
• ctc_loss (FourierConditioner): Encodes CTC loss values. For inference, always use low values (e.g., 0.0 or 1.0).
• dnsmos_ovrl (FourierConditioner): A MOS score for output audio. Set to 4.0 for very clean, neutral English speech. Otherwise, use 'unconditional'.
• speaker_noised (IntegerConditioner): Set to 1 (True) if the input speaker embedding is noisy. This enables the model to denoise the embedding. This is particularly useful if cloning outputs sound echo-y or distorted.