DOCSEARCHbyAlgolia
Submit Docs

Insanely Fast Whisper API

repository·main·Indexed 43 days ago

https://github.com/interfazeai/insanely-fast-whisper-api

A high-speed audio transcription and translation API using OpenAI's Whisper Large v3, optimized with Flash Attention 2 and batching. Version 0.2.0 supports synchronous and asynchronous processing via webhooks, speaker diarization via Hugging Face tokens, and is pre-configured for deployment on Fly.io using Docker.

Tokens
1.9K
Snippets
8
Records
9
Agent score
6%

What's inside Insanely Fast Whisper API

  2. Run the API locally

    main

    To run the API on your local machine, follow these steps to set up the environment and dependencies (requires Python and Poetry):

    1. Clone the repo and enter the directory.
    2. Install PyTorch: pip3 install torch torchvision torchaudio.
    3. Install build dependencies for FlashAttention: pip3 install -U wheel && pip install ninja packaging.
    4. Install FlashAttention: pip3 install flash-attn --no-build-isolation.
    5. (Optional) Generate requirements: poetry export --output requirements.txt.
    6. Set up a virtual environment using your Python path.
    7. Install requirements: poetry install.
    8. Start the server: uvicorn app.app:app --reload.
    # Full local setup sequence
git clone https://github.com/jigsawstack/insanely-fast-whisper-api.git
cd insanely-fast-whisper-api
pip3 install torch torchvision torchaudio
pip3 install -U wheel && pip install ninja packaging
pip3 install flash-attn --no-build-isolation
poetry install
uvicorn app.app:app --reload

  3. Shut down Fly machines programmatically to save costs

    main

    Fly machines are billed by the second. To avoid idling costs, you can programmatically shut down a machine by sending a POST request to the Fly Machines API. This is useful once your processing tasks are complete.

    POST https://api.machines.dev/v1/apps/<app_name>/machines/<machine_id>/stop
Authorization: Bearer <fly_token>

  4. Deploy to Fly.io using Fly Machines

    main

    To deploy the API on Fly.io, ensure you have access to Fly GPUs and the flyctl CLI installed.

    1. Clone the repository and enter the root directory.
    2. (Optional) Rename the app name in fly.toml.
    3. (Optional) Remove image = 'yoeven/insanely-fast-whisper-api:latest' from fly.toml if you prefer to rebuild from the Dockerfile.
    4. Run fly launch and follow the prompts (select y to copy configuration from the repo and n to proceed without tweaking settings).
    5. Set required secrets for authentication and speaker diarization using fly secrets set.

    To enable speaker diarization, you must first accept the user conditions for pyannote/segmentation-3.0 and pyannote/speaker-diarization-3.1 on Hugging Face, then provide your Hugging Face access token.

    # Set authentication and diarization tokens
fly secrets set ADMIN_KEY=<your_token> HF_TOKEN=<your_hf_key>

# Verify secrets
fly secrets list

# View real-time logs
fly logs -a insanely-fast-whisper-api

  5. Authenticate with the API using an Admin Key

    main

    If the ADMIN_KEY environment variable is set on the server, all requests must include the key in the headers to be authorized.

    Header Key: x-admin-api-key

    If the key is missing or incorrect, the server returns a 401 Unauthorized response.

    curl -X POST http://api-url/ \
  -H "x-admin-api-key: YOUR_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/audio.mp3"}'

  6. Manage transcription tasks

    main

    The API provides endpoints to monitor and control transcription jobs:

    • GET /tasks: Retrieve all active transcription tasks (both async background tasks and ongoing tasks).
    • GET /status/{task_id}: Get the status of a specific task. Note: Completed tasks are removed from the list and may return an error if queried after completion.
    • DELETE /cancel/{task_id}: Cancel an asynchronous background task. Only tasks created with is_async: true can be cancelled.

  7. Transcribe or Translate audio via POST /

    main

    The primary endpoint for processing audio. It supports both synchronous transcription and asynchronous background tasks with webhooks.

    Request Body (JSON):

    • url (Required): URL of the audio file.
    • task: transcribe (default) or translate.
    • language: Language code (e.g., en) or None for auto-detection.
    • batch_size: Number of parallel batches (default: 64). Reduce this if encountering Out of Memory (OOM) errors.
    • timestamp: chunk (default) or word.
    • diarise_audio: Boolean to enable speaker diarization (requires HF_TOKEN).
    • webhook: Object containing url and optional header to receive results.
    • is_async: Boolean. If true, the task runs in the background and results are sent via webhook.
    • managed_task_id: Custom Task ID (defaults to a generated UUID v4).
    {
  "url": "https://example.com/audio.mp3",
  "task": "transcribe",
  "language": "en",
  "batch_size": 32,
  "is_async": true,
  "webhook": {
    "url": "https://your-app.com/webhook"
  }
}

  8. Manage background tasks via /tasks, /status/{task_id}, and /cancel/{task_id}

    main

    When using is_async=true, the API provides endpoints to monitor and control the lifecycle of the background task.

    List all tasks

    GET /tasks returns a list of all current task_ids being tracked by the server.

    Check task status

    GET /status/{task_id} returns the current state of a specific task:

    • {"status": "processing"}: The task is still running.
    • {"status": "completed", "output": ...}: The task finished successfully; the output contains the transcription results.
    • Returns 404 if the task_id does not exist.

    Cancel a task

    DELETE /cancel/{task_id} attempts to stop a running background task. If the task is already finished, it returns the result. If it is not a background task, it returns a 400 error.

    # Check status
curl -X GET http://api-url/status/uuid-task-id

# Cancel task
curl -X DELETE http://api-url/cancel/uuid-task-id

  9. Configure Webhook payloads for async tasks

    main

    When is_async is set to true, the API requires a webhook object in the request body. Once the transcription is complete, the server will POST a JSON payload to the provided url.

    Webhook Payload Schema

    • status: Either completed or error.
    • task_id: The ID of the processed task.
    • output: The transcription results (if status is completed).
    • error: The error message string (if status is error).
    • fly_machine_id: (Optional) Included if running on Fly.io.
    // Example successful webhook payload
{
  "output": {
    "text": "Hello world",
    "chunks": [...],
    "speakers": [...] 
  },
  "status": "completed",
  "task_id": "uuid-string",
  "fly_machine_id": "machine-id"
}