DOCSEARCHbyAlgolia
Submit Docs

Microsoft Cognitive Services Speech SDK Samples

repository·master·Indexed 25 days ago

https://github.com/azure-samples/cognitive-services-speech-sdk

A collection of samples for the Microsoft Cognitive Services Speech SDK to help developers implement speech recognition, synthesis, and translation. Includes C++ quickstarts for Linux, macOS, and Windows, covering scenarios such as recognizing speech from a microphone or file and performing text-to-speech synthesis.

Tokens
103.3K
Snippets
141
Records
561
Agent score
85%

What's inside cognitive-services-speech-sdk

  1. Overview of the Ingestion Client

    master
    The Ingestion Client is an automated, scalable transcription solution that monitors an Azure Storage container. When new audio files land in the dedicated container, the client automatically triggers transcription using the Azure Speech service batch pipeline. It can also be configured to perform additional text analytics, such as sentiment analysis, PII redaction, and mapping transcripts to a SQL database.

  2. Overview of SpeechSDKRemoteServiceGRPC architecture

    master

    The SpeechSDKRemoteServiceGRPC project implements a middle-tier service in a multi-tier architecture. It acts as a bridge between an edge client device and the Azure Speech Service using GRPC as the transport layer.

    Workflow:

    1. Edge Client Device: Sends audio data via GRPC.
    2. Middle Tier Service: Receives audio, processes it using the Azure Speech SDK.
    3. Azure Speech SDK/Service: Performs the actual speech recognition.

    The architecture includes a robust protocol to handle link failures by allowing the client to resend unacknowledged audio data.

  3. Overview of Embedded and Hybrid Speech in C++

    master

    This sample project demonstrates two speech modes using the Speech SDK for C++:

    1. Embedded Speech: Enables offline (on-device) speech recognition, synthesis, and translation.
    2. Hybrid Speech: Uses cloud speech services by default and falls back to embedded speech if cloud connectivity is limited.

    Audio Format Requirements for Embedded Speech

    Embedded speech recognition and translation only support audio in the following format:

    • Single channel
    • 8000 or 16000 Hz sample rate
    • 16-bit little-endian signed integer samples

  5. Overview of SpeechSDKRemoteClientCore

    master
    The SpeechSDKRemoteClientCore is a core component designed for building a middle-tier service that processes audio from an Edge Client device. It serves as a shared library for different client transport implementations (such as WebSockets or gRPC) to handle audio streaming and communication with a middle-tier service, which then interacts with the Azure Speech Service.

  7. Implement Multi-tier Speech SDK Architecture

    master

    The Multi-tier Speech SDK Sample provides a pattern for implementing Azure Speech SDK functionality where audio capture and speech recognition results are processed on a middle-tier server instead of the client device. This is useful when:

    • The Speech SDK cannot run directly on the client device.
    • You need to apply additional business logic or processing on a middle tier.
    • You need to maintain persistent Speech SDK connections across multiple client sessions.
    • You require robust audio transport over unreliable networks.

  8. Advanced Objective-C iOS Speech Techniques

    master

    The Objective-C samples for iOS provide implementations for several advanced speech scenarios:

    Connection Management

    The connection-sample demonstrates how to establish a connection to the speech service independently of starting a recognition task. This technique is used to reduce latency for the initial recognition task (e.g., recognition from a microphone).

    Speech Recognition and Translation

    The speech-samples app demonstrates:

    • Using phrase hints to improve recognition accuracy.
    • Implementing continuous recognition.
    • Using push and pull audio streams.

    Speech Synthesis (TTS)

    The synthesis-samples app demonstrates:

    • Using push and pull audio streams (or audio data streams) to fetch synthesized audio streamingly to minimize latency.
    • Using event handlers to manage synthesis events.

  9. Prerequisites and Authentication for Batch Synthesis

    master

    Before using the Batch Synthesis API, ensure you meet the following requirements:

    Requirements

    • Python: Version 3.8 or higher.
    • Subscription: Batch synthesis is only available for paid subscriptions; free subscriptions are not supported.
    • Custom Voices (Optional): You need the relationship between custom voice names and deployment ID.
    • Azure Blob Storage (Optional): The URI of a writable Azure blob container if you wish to store audio files in your own storage.

    Authentication Options

    • Passwordless (Recommended): Use the azure-identity library. Your Microsoft Entra user account must be assigned the Cognitive Services User or Cognitive Services Speech User role.
    • API Key: Use the subscription key found in the "Keys and Endpoint" tab of your Azure AI Speech resource in the Azure Portal.

  10. Access Shared Content for Language Learning Samples

    master
    The scenarios/csharp/sharedcontent/console directory contains shared assets used by the Language Learning samples. This content is utilized by both the .NET Framework (Windows) and .NET Core (Windows or Linux) implementations. To use these samples, refer to their respective documentation for setup and execution instructions.

  11. TTS Text Stream functionality overview

    master

    The TTS Text Stream sample demonstrates the following capabilities using the Speech SDK:

    • Incremental Text Streaming: Uses SpeechSynthesisRequestInputType.TextStream to push text to Azure TTS in small chunks.
    • Audio Collection: Collects streamed PCM audio from an AudioDataStream and writes the result to a WAV file.
    • Endpoint Usage: Utilizes the websocket v2 endpoint required for text stream APIs.

  12. Use Batch Avatar Synthesis via REST API

    master

    The Batch avatar synthesis API (Preview) allows for asynchronous synthesis of talking avatars to generate video content from text input.

    Important Implementation Note: These samples do NOT require the installation of the Cognitive Service Speech SDK. Instead, they interact with the Batch avatar synthesis functionality by calling the REST API directly.

    For real-time avatar synthesis (as opposed to batch), refer to the real-time avatar synthesis documentation.