DOCSEARCHbyAlgolia
Submit Docs

ttsfm Documentation

repository·main·Indexed 20 days ago

https://github.com/dbccccccc/ttsfm

An OpenAI-compatible text-to-speech API service and client. It provides a reverse-engineered implementation of the openai.fm service, featuring synchronous (TTSClient) and asynchronous (AsyncTTSClient) workflows, WebSocket support for real-time streaming, and Docker deployment options in Full (with ffmpeg for speed adjustment and format conversion) and Slim variants.

Tokens
14K
Snippets
46
Records
61
Agent score
72%

What's inside ttsfm

  1. Choosing between TTSClient and AsyncTTSClient

    main

    TTSFM provides two primary client types depending on your integration needs:

    • TTSClient (Synchronous): The standard client used by the internal Flask REST endpoints and the WebSocket handler. It is designed for isolated, per-request execution to prevent session races.
    • AsyncTTSClient (Asynchronous): Designed for external consumers who require fully asynchronous workflows for high-concurrency or non-blocking applications.

  3. Choose between Full and Slim Docker variants

    main

    TTSFM provides two distinct Docker image variants depending on your requirements for audio processing capabilities and image size.

    Full Variant

    Use this variant if you require advanced audio features. It includes ffmpeg and supports:

    • MP3 auto-combine (concatenation)
    • Speed adjustment (0.25x - 4.0x)
    • Format conversion
    • Multi-platform support (linux/amd64, linux/arm64)

    Slim Variant

    Use this variant for a smaller footprint where advanced audio processing is not needed. It does not include ffmpeg and has the following limitations:

    • No MP3 auto-combine (only supports WAV auto-combine)
    • No speed adjustment
    • No format conversion
    • Multi-platform support (linux/amd64, linux/arm64)
    • Smoke tests run on port 8001 (Full variant uses 8000)

  4. How TTSFM architecture works

    main

    TTSFM is structured as a multi-layered system designed for both synchronous and asynchronous text-to-speech workflows. The architecture consists of:

    1. Frontend (JS): Includes a Playground UI and a streaming UI that communicates via REST and WebSockets.
    2. Flask REST Endpoints: Handles standard API requests via /api/* and /v1/audio routes.
    3. WebSocket Handler: Manages background tasks and streaming chunk delivery for real-time UI updates.
    4. Upstream Provider: Communicates with the OpenAI.fm upstream service.

    To ensure concurrency safety, the system uses a per-request client model where each request receives an isolated TTSClient instance, preventing shared session races during concurrent operations.

    +----------------+       +--------------------+       +----------------------+
| Frontend (JS)  | <---> | Flask REST Endpoints| <---> | OpenAI.fm upstream   |
| Playground UI  |       | /api/* + /v1/audio  |       | reverse-engineered   |
+----------------+       +--------------------+       +----------------------+
        |                               ^
        v                               |
+----------------+       +--------------------+
| Socket.IO WS   | <---> | WebSocket Handler  |
| streaming UI   |       | (background tasks) |
+----------------+       +--------------------+

  5. Deploy TTSFM via Docker

    main

    TTSFM provides two Docker image variants. The choice depends on whether you need advanced audio processing features like speed adjustment or specific audio formats.

    Full Version (Recommended)

    Includes ffmpeg, supporting all 6 audio formats (MP3, WAV, OPUS, AAC, FLAC, PCM), speed adjustment (0.25x - 4.0x), and automatic MP3/WAV merging for long text.

    docker run -p 8000:8000 dbcccc/ttsfm:latest

    Slim Version

    A minimal image without ffmpeg. It only supports basic TTS, 2 audio formats (MP3, WAV), and long text WAV merging. It does not support speed adjustment, format conversion, or MP3 merging.

    docker run -p 8000:8000 dbcccc/ttsfm:slim

    By default, the container exposes a Web Playground at http://localhost:8000 and an OpenAI-compatible /v1/audio/speech endpoint.

  6. Deploy TTSFM with WebSocket support using Docker

    main

    To enable real-time audio streaming, you must build and run the TTSFM Docker image with WebSocket support enabled. It is recommended to use the ttsfm-websocket tag.

    Build and Run locally

    # Build with WebSocket support
docker build -t ttsfm-websocket .

# Run with WebSocket enabled
docker run -p 8000:8000 \
  -e DEBUG=false \
  ttsfm-websocket

    Production Deployment

    When deploying to a server, ensure you set necessary security environment variables like REQUIRE_API_KEY and TTSFM_API_KEY.

    docker run -d \
  --name ttsfm-container \
  -p 8000:8000 \
  -e REQUIRE_API_KEY=true \
  -e TTSFM_API_KEY=your-secret-key \
  -e DEBUG=false \
  ttsfm-websocket:latest
    docker build -t ttsfm-websocket .

docker run -p 8000:8000 \
  -e DEBUG=false \
  ttsfm-websocket

  7. Use the TTSFM Python SDK

    main

    The TTSClient provides synchronous and asynchronous methods to generate speech. You can specify the text, voice, audio format, and playback speed.

    Note: Speed adjustment requires ffmpeg to be available in your environment.

    from ttsfm import TTSClient, AudioFormat, Voice

client = TTSClient()

# Basic usage
response = client.generate_speech(
    text="来自 TTSFM 的问候！",
    voice=Voice.ALLOY,
    response_format=AudioFormat.MP3,
)
response.save_to_file("hello")  # -> hello.mp3

# With speed adjustment (requires ffmpeg)
response = client.generate_speech(
    text="这段语音会更快！",
    voice=Voice.NOVA,
    response_format=AudioFormat.MP3,
    speed=1.5,  # Range: 0.25 - 4.0
)
response.save_to_file("fast")  # -> fast.mp3
    from ttsfm import TTSClient, AudioFormat, Voice

client = TTSClient()

# 基础用法
response = client.generate_speech(
    text="来自 TTSFM 的问候！",
    voice=Voice.ALLOY,
    response_format=AudioFormat.MP3,
)
response.save_to_file("hello")  # -> hello.mp3

# 使用语速调节（需要 ffmpeg）
response = client.generate_speech(
    text="这段语音会更快！",
    voice=Voice.NOVA,
    response_format=AudioFormat.MP3,
    speed=1.5,  # 1.5 倍速（范围：0.25 - 4.0）
)
response.save_to_file("fast")  # -> fast.mp3

  8. Install TTSFM via pip

    main

    You can install the TTSFM client using pip. Choose between the core client or the version including web and server dependencies.

    • Use ttsfm for the core client.
    • Use ttsfm[web] if you need the core client plus web/server dependencies.
    pip install ttsfm        # core client
pip install ttsfm[web]   # core client + web/server dependencies

  9. Use speed adjustment in TTSFM

    main

    TTSFM supports playback speed adjustment ranging from 0.25x to 4.0x. This feature is implemented using ffmpeg's atempo filter.

    Requirements

    • Docker: Use the Full variant (dbcccc/ttsfm:latest).
    • Python Package: You must have ffmpeg installed on your system (apt-get install ffmpeg on Linux or brew install ffmpeg on Mac).
    • REST API: Use the speed parameter in the /v1/audio/speech endpoint.

    Implementation Details

    • Speed adjustment is applied post-generation.
    • The API response metadata includes speed_applied: true/false to indicate if the adjustment was successful.
    # Python Client Example
from ttsfm import TTSClient, Voice

client = TTSClient()
response = client.generate_speech(
    text="This will be faster!",
    voice=Voice.NOVA,
    speed=1.5,
)
response.save_to_file("fast.mp3")

  10. Install TTSFM via Docker

    main

    TTSFM provides two Docker image variants. The Full variant is recommended as it includes ffmpeg for advanced features like speed adjustment and all audio formats. The Slim variant is a minimal image (~100MB) without ffmpeg.

    Full variant (Recommended)

    Includes ffmpeg for:

    • All 6 audio formats (MP3, WAV, OPUS, AAC, FLAC, PCM)
    • Speed adjustment (0.25x - 4.0x)
    • Format conversion
    • MP3/WAV auto-combine for long text

    Slim variant (~100MB)

    Minimal image without ffmpeg. Supports:

    • Basic TTS functionality
    • 2 audio formats (MP3, WAV only)
    • WAV auto-combine for long text
    • Note: No speed adjustment, no format conversion, and no MP3 auto-combine.
    # Run Full variant
docker run -p 8000:8000 dbcccc/ttsfm:latest

# Run Slim variant
docker run -p 8000:8000 dbcccc/ttsfm:slim

  11. Use the WebSocketTTSClient for real-time speech generation

    main

    The WebSocketTTSClient allows you to generate speech and receive audio chunks in real-time. This reduces perceived latency by allowing playback to start before the entire text is processed.

    Basic Usage

    Initialize the client with your socketUrl and use generateSpeech to trigger the stream. You can provide callbacks for onProgress, onChunk, and onComplete to handle the lifecycle of the generation.

    Advanced Chunk Processing

    You can implement custom logic within the onChunk callback, such as pushing audio data to a playback queue immediately after the first chunk arrives.

    // Initialize WebSocket client
const client = new WebSocketTTSClient({
    socketUrl: 'http://localhost:8000',
    debug: true
});

// Generate speech with streaming
const result = await client.generateSpeech('Hello, WebSocket world!', {
    voice: 'alloy',
    format: 'mp3',
    onProgress: (progress) => {
        console.log(`Progress: ${progress.progress}%`);
    },
    onChunk: (chunk) => {
        console.log(`Received chunk ${chunk.chunkIndex + 1}`);
        // Process audio chunk in real-time
    },
    onComplete: (result) => {
        console.log('Generation complete!');
        // Play or download the combined audio
    }
});

  12. Install the TTSFM Python package

    main

    You can install the TTSFM client via pip. Choose between the core client or the version including Web/server dependencies.

    • Core client: For basic SDK usage.
    • Web/server dependencies: Includes extra requirements for web-related functionality.
    pip install ttsfm        # Core client
pip install ttsfm[web]   # Core client + Web/server dependencies
    pip install ttsfm        # 核心客户端
pip install ttsfm[web]   # 核心客户端 + Web/服务端依赖