DOCSEARCHbyAlgolia
Submit Docs

Fast-Powerful-Whisper-AI-Services-API

repository·main·Indexed 19 days ago

https://github.com/evil0ctal/fast-powerful-whisper-ai-services-api

A high-performance, asynchronous API service for speech-to-text tasks using OpenAI Whisper and Faster Whisper. It features an asynchronous model pool for multi-GPU load balancing, task management for transcription and translation, and built-in web crawlers for Douyin and TikTok. The service supports priority levels, callback URLs, subtitle generation (.srt, .vtt), and ChatGPT integration for summarization and analysis.

Tokens
7.9K
Snippets
18
Records
37
Agent score
67%

What's inside fast-powerful-whisper-ai-services-api

  1. Core Features and Capabilities

    main

    The Fast-Powerful-Whisper-AI-Services-API is a high-performance, asynchronous speech recognition service built on OpenAI Whisper and Faster Whisper.

    Key Capabilities:

    • Asynchronous Model Pool: Supports multi-instance concurrent processing. In multi-GPU environments, it dynamically assigns models to GPUs to balance load. (Note: Concurrency is not available on single-GPU setups).
    • Task Management: Supports creating tasks via media file upload (file_upload) or media links (file_url).
    • Task Types: Supports transcribe (transcription) and translate (automatic translation).
    • Priority Levels: Tasks can be assigned high, normal, or low priority.
    • Web Crawlers: Built-in support for processing videos from Douyin and TikTok via links.
    • Callbacks: Supports callback_url to receive an HTTP POST request upon task completion.
    • Subtitle Generation: Can generate .srt or .vtt subtitle files from a task_id.
    • ChatGPT Integration: Allows using task data to interact with ChatGPT for summarization and analysis.

  2. Understand the API Layer and Endpoints

    main

    The API layer is built with FastAPI and manages the primary interface for interacting with the service. It uses Pydantic models (defined in APIResponseModel.py) to ensure consistent response structures.

    Key routing modules include:

    • health_check.py: Monitors system status.
    • whisper_tasks.py: Manages Whisper-related tasks, including creation, querying, and deletion.
    • work_flows.py: Provides CRUD interfaces for managing automated workflows.

  3. Understand the project structure

    main

    The project is organized into several functional layers to separate API concerns, business logic, and data management:

    • app/api/: The API layer. Contains models/ for request/response validation (e.g., WhisperTaskRequest.py, ChatGPTTaskRequest.py) and routers/ for endpoint definitions (e.g., whisper_tasks.py, chatgpt_tasks.py).
    • app/crawlers/: Web crawling modules for specific platforms like douyin and tiktok.
    • app/database/: Data persistence layer. Includes models/ for tasks and workflows, and DatabaseManager.py for connection handling.
    • app/services/: The service layer containing core logic for whisper_service.py, workflow_service.py, and callback_service.py.
    • app/processors/: Handles the execution of tasks and workflows via task_processor.py and workflow_processor.py.
    • app/model_pool/: Manages asynchronous model pooling via AsyncModelPool.py.
    • app/workflows/: Contains modular components for building complex workflows.
    • config/: Contains settings.py for application configuration.
    • temp_files/ & log_files/: Directories for temporary processing files and application logs.
    • WhisperServiceAPI.db: The default SQLite database file.

  4. Understand the Project Directory Structure

    main

    The project is organized into several functional modules designed for high-performance asynchronous processing of Whisper AI tasks. Understanding these modules helps in identifying where to find API endpoints, database logic, model management, and background processing:

    • app/api/: FastAPI-based routing layer. Includes health_check.py, whisper_tasks.py, and work_flows.py.
    • app/crawlers/: Asynchronous crawlers (using httpx) for platforms like Douyin and TikTok.
    • app/database/: SQLAlchemy-based asynchronous database management (supports MySQL and SQLite).
    • app/http_client/: Robust asynchronous HTTP client (AsyncHttpClient) with retry, backoff, and proxy support.
    • app/model_pool/: Manages GPU/CPU model instances using AsyncModelPool to optimize hardware utilization.
    • app/processors/: Background task processing logic (task_processor.py and workflow_processor.py) using asyncio and thread pools.
    • app/services/: High-level service logic including WhisperService (audio extraction/transcription) and CallbackService (webhook notifications).
    • app/utils/: Utility modules for asynchronous file operations and concurrent-safe logging.
    • app/workflows/: Framework for defining custom task component flows.
    • config/settings.py: Centralized configuration management via environment variables.

  5. Manage Database Operations with DatabaseManager

    main

    The DatabaseManager in app/database/ provides asynchronous CRUD operations for tasks and workflows using SQLAlchemy. It supports MySQL and SQLite and features automatic connection retries and table initialization.

    Key capabilities include:

    • Task Management: Methods like add_task, get_task, update_task, and delete_task for managing Whisper tasks.
    • Flexible Querying: query_tasks supports pagination and complex filtering via an internal _build_query_conditions builder.
    • Callback Updates: update_task_callback_status specifically manages the recording of webhook response statuses.
    • Workflow Management: Support for creating and managing Workflow objects, including associated tasks and notifications.

  6. Process Whisper tasks with TaskProcessor

    main

    The task_processor.py (in app/processors/) handles the background execution of Whisper tasks using a multi-queue design and asyncio combined with thread pools.

    Key mechanisms:

    • Priority Scheduling: Uses multiple queues (e.g., for cleanup, callbacks) to ensure high-priority tasks are processed first.
    • Parallelism: _process_multiple_tasks utilizes a thread pool to handle multiple tasks concurrently.
    • Execution Loop: The run_loop method starts the background event loop that continuously monitors and processes task queues.

  7. Optimize Hardware with AsyncModelPool

    main

    The AsyncModelPool in app/model_pool/ manages the lifecycle of GPU and CPU model instances (like Whisper) to maximize hardware efficiency and prevent resource contention.

    Core Logic:

    • Device Allocation: Automatically assigns devices based on available GPU/CPU resources and limits the number of instances per GPU.
    • Instance Management: Uses get_model to acquire an instance and return_model to release it back to the pool. It supports both reusing existing instances and dynamic creation.
    • Concurrency: Designed as a thread-safe singleton to ensure safe access in multi-threaded/asynchronous environments.
    • Health Monitoring: Includes _is_model_healthy and _destroy_model to clean up corrupted instances and prevent memory leaks.

  8. Process Tasks via TaskProcessor and WorkflowProcessor

    main

    Background processing is handled by two main components in app/processors/:

    • task_processor.py: Manages Whisper transcription tasks. It uses a multi-queue design (separating tasks like cleanup or callbacks) and supports priority scheduling. It combines asyncio for event loop management and concurrent.futures (thread pools) for parallel task execution.
    • workflow_processor.py: Manages complex, multi-step workflows. It allows for custom task dependencies, conditional logic, and automated scheduling.

  9. How to create a custom crawler platform

    main

    To add a new crawler to the service, follow these steps:

    1. Create a new directory: Under the app/crawlers/platforms/ folder, create a new directory for your specific platform.
    2. Implement crawler logic: Write your crawler code within that new directory. For reference, you can use the existing douyin folder as a demonstration of how to utilize the AsyncHttpClient module for asynchronous data crawling.
    3. Expose via API: To make your crawler accessible via HTTP requests, you must define and customize new routes in app/api/router.py that call your crawler implementation.

  10. Install FFmpeg

    main

    FFmpeg is required for audio and video transcoding. Install it using the command corresponding to your operating system:

    • Ubuntu/Debian: sudo apt update && sudo apt install ffmpeg
    • Arch Linux: sudo pacman -S ffmpeg
    • MacOS (Homebrew): brew install ffmpeg
    • Windows (Chocolatey): choco install ffmpeg
    • Windows (Scoop): scoop install ffmpeg
    # Example for Ubuntu
sudo apt update && sudo apt install ffmpeg

  11. Quick Deployment Guide

    main

    Follow these steps to deploy the Fast-Powerful-Whisper-AI-Services-API locally. The project requires Python (3.12 recommended or >=3.8) and FFmpeg for media processing.

    1. Clone the repository

    git clone https://github.com/Evil0ctal/Fast-Powerful-Whisper-AI-Services-API.git

    2. Install FFmpeg

    Install FFmpeg based on your operating system:

    • Ubuntu/Debian: sudo apt update && sudo apt install ffmpeg
    • Arch Linux: sudo pacman -S ffmpeg
    • MacOS: brew install ffmpeg
    • Windows (Chocolatey): choco install ffmpeg
    • Windows (Scoop): scoop install ffmpeg

    3. Install CUDA and PyTorch (Optional for GPU acceleration)

    If you have an NVIDIA GPU, install the CUDA Toolkit and then install PyTorch with CUDA support:

    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

    4. Install Dependencies and Start

    Navigate to the project directory, install the requirements, and run the API:

    pip install -r requirements.txt
python3 start.py

    After starting, access the interactive Swagger UI at http://127.0.0.1/.

    git clone https://github.com/Evil0ctal/Fast-Powerful-Whisper-AI-Services-API.git
# ... follow subsequent steps in the guide
python3 start.py