DOCSEARCHbyAlgolia
Submit Docs

Speechmatics Documentation

website·Indexed 37 days ago

https://beta-docs.speechmatics.com

Technical documentation for Speechmatics ASR (Automatic Speech Recognition) services, including the REST and Realtime APIs, Management API, and on-premise deployment options via CPU/GPU containers and Virtual Appliances. Covers transcription job management, API key authentication, workspace administration, and Kubernetes deployment.

Tokens
91.7K
Snippets
439
Records
642
Agent score
50%

What's inside Speechmatics

  1. Overview of Agent STT for voice agents

    Agent STT is a SaaS-only transcription service designed for voice agents. It provides turn-based, speaker-labelled transcription with punctuation, suitable for LLM consumption. Key features include:

    • Turn detection: Identifying when a speaker has finished talking.
    • Intelligent segmentation: Grouping words into speaker-attributed segments.
    • Diarization: Labeling different speakers.
    • Speech signals: Notifications for the start and end of speech.

  2. Overview of the Voice Agent API

    The Voice Agent API is a preview WebSocket API designed for building voice agents. It provides speaker-labelled, turn-based transcription that is punctuated and ready for LLM consumption. Turn detection is handled server-side based on the selected profile.

    Important Preview Constraints:

    • Not for live production traffic.
    • No uptime or performance SLAs.
    • No data residency guarantees (processing may occur in US and EU regions).
    • Features may change or be cancelled.

  3. Overview of the Language ID Container

    The Speechmatics Language ID Container predicts the most likely, predominant language spoken in a media file. This allows users to programmatically select the correct transcriber when the source language of a file is unknown. For high accuracy, it is recommended that audio files contain at least 60 seconds of speech.

  4. Overview of the Batch persistent worker

    The Batch persistent worker (also known as the HTTP batch worker), available from version 15.7.0, is a long-running transcription service. Unlike standard batch containers that exit after one job, the persistent worker loads ASR models once at startup and accepts multiple jobs via an HTTP API. This eliminates per-job cold starts and allows for concurrent processing using the --parallel flag to manage processing units (engines).

  5. Overview of Speechmatics integrations and SDKs

    Speechmatics provides multiple ways to integrate Speech-to-Text (STT) and Text-to-Speech (TTS) capabilities into applications:

    • Voice Agent Orchestrators: Frameworks for building low-latency voice agents, including Vapi (no-code/prototyping), LiveKit (WebRTC-based for engineers), and Pipecat (full pipeline control for power builders).
    • No-code Integrations: Zapier for automating transcription workflows.
    • SDKs: Direct API access via Python (Agent STT, Realtime, Batch, TTS), JavaScript (Realtime, Batch), .NET (Realtime), and Rust (Realtime and Batch).

  6. Deploy the Omni model using Batch Containers

    The Omni model uses a two-container architecture: a GPU inference server (Triton) and a transcriber container. The containers are designed to spin up for a job and spin down upon completion. To deploy, first create a dedicated Docker network, then launch the inference server and the transcriber.
    # 1. Create the network
docker network create -d bridge transcription

# 2. Run the Batch Inference Server (Triton)
docker run --rm -it \
  --network=transcription \
  --name=sm-triton-batch \
  -v $PWD/license.json:/license.json \
  --gpus '"device=0"' \
  -e CUDA_VISIBLE_DEVICES \
  -e SM_BATCH_MODE=true \
  -p 8001:8001 \
  speechmaticspublic.azurecr.io/demo/sm-gpu-inference-server-aed:0.4.0

  7. Choose between Speechmatics Cloud and On-prem deployments

    Speechmatics offers two primary deployment methods:

    1. Cloud (SaaS): Fully managed speech-to-text and translation capabilities. Best for easy scalability. Access is managed via the Speechmatics Portal.
    2. On-prem: Deploy containers in your own environment for maximum control over data and deployment. Options include CPU-based containers, GPU-based containers, Kubernetes deployments, Language ID containers, and Translation containers.

  8. Organize work using Projects

    Projects are organizational units within a workspace used to scope API keys, isolate transcripts, and track usage. Key characteristics include:

    • API Key Scoping: API keys are tied to a single project. A key created in one project cannot access transcripts from another.
    • Access: Any member of the workspace can access any project within that workspace.
    • Limits: A workspace can have up to 1000 projects. Each workspace starts with a default project that cannot be deleted.
    • Billing: While usage can be viewed per project, billing and usage are not segmented by project at the workspace level.

  9. Analyze sentiment data in Speechmatics transcripts

    Sentiment data is structured to provide both a high-level overview and granular detail. The summary object includes:

    • overall: Counts of positive_count, negative_count, and neutral_count for the entire file.
    • speakers: An array of objects containing the speaker label and their specific sentiment counts.
    • channels: An array of objects containing the channel label and its specific sentiment counts.

  10. Understand Speechmatics Workspace organization

    A workspace is the top-level container for a team's use of Speechmatics. It serves as the primary boundary for members, projects, API keys, billing, and usage. All actions performed in the portal or via the API occur within the context of a single workspace.

    Key components of a workspace include:

    • Members: Users with assigned roles.
    • Projects: Organizational units used to scope API keys and track usage separately.
    • API Keys & Management Tokens: API keys are scoped to specific projects, while management tokens are scoped to the entire workspace.
    • Billing and Usage: Pricing plans, payment methods, credits, and aggregated usage are managed at the workspace level.

  11. Choose a Speech-to-Text processing mode

    Speechmatics offers three primary modes for converting audio to text depending on the latency and use-case requirements:

    1. Realtime processing: Best for live broadcasts, webinars, or AI agents where instant updates are required via a WebSocket API.
    2. Batch processing: Best for pre-recorded files (meetings, interviews, archives). Jobs are processed asynchronously by default, though synchronous transcription is available for single requests.
    3. Agent STT: Specialized for voice agents and conversational AI. It provides speaker-labeled, punctuated segments and turn-detection (start/end markers) to signal when an LLM should respond.