Chatterbox TTS Server

Chatterbox TTS Server is a self-hosted Text-to-Speech (TTS) solution that provides an OpenAI-compatible API and a modern Web UI. It is built around Resemble AI's Chatterbox open-source model family.

Key capabilities include:

• Model Support: Supports the original high-quality model, Multilingual support (23 languages), and Chatterbox-Turbo (a high-throughput 350M-parameter model).
• Expressive Audio: Chatterbox-Turbo supports paralinguistic tags like [laugh], [cough], and [chuckle].
• Advanced Features: Voice cloning, large text processing via intelligent chunking, audiobook generation, and reproducible voices using a generation seed.
• Hardware Acceleration: Optimized for NVIDIA (CUDA), AMD (ROCm), and Apple Silicon (MPS) GPUs, with a CPU fallback.
• Compatibility: Provides an OpenAI-compatible API, making it easy to swap into existing workflows designed for OpenAI's TTS.

The project is organized into several key functional areas:

• Core Logic: server.py (FastAPI application), engine.py (model loading and generation), config.py (configuration management), and models.py (API validation).
• Web UI: Located in the ui/ directory, containing index.html, script.js, and presets.yaml.
• Data & Assets:
  • config.yaml: The primary configuration file.
  • outputs/: Default directory for generated audio files.
  • reference_audio/: Directory for user-uploaded audio used in voice cloning.
  • voices/: Directory for predefined voice audio files.
  • model_cache/: Local cache for models downloaded via download_model.py.

Chatterbox-TTS-Server/
│
├── config.py             # Manages config.yaml, default values, accessors
├── config.yaml           # PRIMARY configuration file (created/managed by server)
├── docker-compose.yml    # Docker Compose setup for containerized deployment
├── Dockerfile            # Docker image definition
├── documentation.md      # This comprehensive documentation file
├── download_model.py     # Utility to download specific model files to a local cache
├── engine.py             # Core model loading (from_pretrained) & generation logic
├── models.py             # Pydantic models for API request validation and structure
├── README.md             # Project summary and quick start guide
├── requirements.txt      # Python package dependencies
├── server.py             # Main FastAPI application, API endpoints, UI routes
│
├── ui/                   # Contains all files for the Web User Interface
│   ├── index.html        # Main HTML template for the UI
│   ├── presets.yaml      # Predefined examples for TTS generation, loaded by the UI
│   └── script.js         # Frontend JavaScript for UI interactivity and API communication
│
├── model_cache/          # Default directory for download_model.py script
├── outputs/              # Default directory for audio files saved from UI or API
├── reference_audio/      # Default directory for user-uploaded reference audio files for voice cloning
└── voices/               # Default directory for predefined voice audio files

The server operates through the interaction of several components:

1. User Interface/Client: Interacts with the FastAPI server via the Web UI or direct API calls.
2. FastAPI Server (server.py): Serves the Web UI, handles API requests, and manages configuration via config.py.
3. TTS Engine (engine.py): Receives parameters and text chunks from the server to perform actual generation using the ChatterboxTTS model (loaded from Hugging Face).
4. Utilities: Handles chunking logic, audio processing (using soundfile and librosa), and file management.
5. Backend Dependencies: Relies on PyTorch and CUDA for hardware acceleration.

┌───────────────────┐      ┌───────────────────┐      ┌───────────────────────────┐      ┌───────────────────┐
│ User (Web UI /    │────→ │ FastAPI Server    │────→ │ TTS Engine (engine.py)    │────→ │ ChatterboxTTS     │
│ API Client)       │      │ (server.py)       │      │ (Handles Chunks/Params)   │      │ (from HF Hub)     │
└───────────────────┘      └─────────┬─────────┘      └──────┬─────────┬──────────┘      └─────────┬─────────┘
      ↑                            │                      │ Calls   │                            │ (Uses PyTorch)
      │ (Serves UI,               │ Uses                 │         │                            │
      │  API data)                ▼                      ▼         ▼                            │
      └───────────────── │ Configuration     │ ←─ │ config.yaml     │  │ Utilities         │      │
                         │ (config.py)       │  └───────────────────┘  │ (utils.py)        │      │
                         └───────────────────┘                         │ - Chunking Logic  │      │
                                   ▲                                   │ - Audio Proc.     │      │
                                   │ Uses                              │ - File Handling   │      │
                                   │                                   └──────┬────────────┘      │
┌───────────────────┐      ┌───────────────────┐      ┌───────────────────┐      │                   │
│ Web UI Files      │ ←─── │ API Data / HTML   │      │ Audio Libraries   │←─────┘                   │
│ (ui/*)            │      │ (via server.py)   │      │ (soundfile, librosa)│                       ▼
└───────────────────┘      └───────────────────┘      └───────────────────┘                 ┌───────────────────┐
                                                                                        │ PyTorch / CUDA    │
                                                                                        └───────────────────┘