DOCSEARCHbyAlgolia
Submit Docs

RTZR STT OpenAPI Documentation

website·Indexed 32 days ago

https://developers.rtzr.ai/docs/

Developer guides for RTZR's Speech-to-Text (STT) services, featuring General STT REST APIs for audio files and Streaming STT via gRPC and WebSocket. Includes documentation on the Sommers and Korean Fine-tuned Whisper models, speaker diarization, keyword boosting, PII and profanity filtering, and authentication using OAuth2 and JWT.

Tokens
13.9K
Snippets
54
Records
89
Agent score
62%

What's inside RTZR

  1. Overview of General STT API

    The General STT API is an HTTP-based REST API that converts audio files into text. It supports the following audio formats: mp4, m4a, mp3, amr, flac, and wav. Access requires a JWT token issued via the authentication guide.

  2. Understand RTZR STT concurrency-based rate limits

    RTZR STT OpenAPI controls service usage based on concurrent throughput rather than time-based limits (e.g., requests per second). Limits are applied at the organization (account) level and differ between File-based STT and Streaming STT.

  3. Extract insights from STT transcription results

    The Insight feature allows you to extract specific information from transcription results based on a designated perspective. This feature requires an 'enterprise' license. If keywords are configured for keyword boosting, they are also used as hints during the insight generation process.

  4. Streaming STT Overview and Connection Limits

    The Streaming STT API converts real-time audio streams into text. It supports two connection methods: gRPC and WebSocket.

    Critical Constraints:

    • Concurrent Channels: The number of simultaneous streaming connections is subject to the platform's throughput limit policy.
    • Idle Timeout: If no audio chunks are transmitted for 60 seconds after a connection is established, the server may terminate the connection.

  5. Inquire about On-premise or Enterprise plans

    For technical or pricing inquiries regarding On-premise deployment, Enterprise plans, or high-volume usage, refer to the plans section of the pricing page.

  6. Remove filler words and repetitions using the Disfluency Filter

    The Disfluency Filter removes meaningless expressions (filler words) and redundant repetitions from transcribed speech to provide more concise and readable text. Examples of filtered content include hesitation markers like '음' (um), '뭐' (well), and '아' (ah), as well as stuttered or repeated words (e.g., '응응응' becomes '응').

  7. Use FINALIZE Command for AI Voice Agents

    For AI Voice Agents or call bots using their own VAD (Voice Activity Detection) or turn detectors, use the FINALIZE command to force the current utterance to be finalized. This allows the agent to receive the final transcript and generate a response faster without waiting for the server's end-point detection.

    • Behavior: FINALIZE does not close the connection; you can continue sending audio for the next turn on the same connection.
    • Termination: To end the conversation, half-close the gRPC request stream or send an EOS text message via WebSocket.
    • Important for OGG_OPUS: If using container-based encoding like OGG_OPUS, you must start the next utterance with a new container header after a FINALIZE command.

  8. Authenticate with RTZR OpenAPI to get an access token

    To authenticate and obtain an access token for gRPC calls, send a POST request to https://openapi.vito.ai/v1/authenticate with the client_id and client_secret in the request body as form data. The API returns a JSON response containing the access_token.
    public static String getAccessToken() throws IOException {
    OkHttpClient client = new OkHttpClient();
    RequestBody formBody = new FormBody.Builder()
            .add("client_id", "{RTZR_CLIENT_ID}")
            .add("client_secret", "{RTZR_CLIENT_SECRET}")
            .build();
    Request request = new Request.Builder()
            .url("https://openapi.vito.ai/v1/authenticate")
            .post(formBody)
            .build();
    Response response = client.newCall(request).execute();
    ObjectMapper objectMapper = new ObjectMapper();
    HashMap<String, String> map = objectMapper.readValue(response.body().string(), HashMap.class);
    return map.get("access_token");
}

  9. Implement streaming STT via WebSocket

    To implement streaming STT, establish a WebSocket connection to wss://openapi.vito.ai/v1/transcribe:streaming. The connection requires an authentication token passed in the Authorization: Bearer {token} header. Audio data should be sent as binary messages. To signal the end of a stream, send a text message containing EOS.
    // Example connection URL with parameters
u := url.URL{Scheme: "wss", Host: "openapi.vito.ai", Path: "/v1/transcribe:streaming"}
query := u.Query()
query.Set("sample_rate", "8000")
query.Set("encoding", "LINEAR16")
query.Set("use_itn", "true")
query.Set("use_disfluency_filter", "true")
query.Set("use_profanity_filter", "false")
u.RawQuery = query.Encode()

  10. Choose between Streaming STT and File STT

    While the Streaming STT API can be used to convert audio files to text, it is recommended to use the standard File STT API for non-real-time audio files as it is more convenient for that specific use case.

  11. Choose between General STT and Streaming STT integration

    ReturnZero STT offers two primary integration methods based on the required latency and data flow:

    1. General STT: Used for uploading files and checking results via asynchronous polling (near real-time).
    2. Streaming STT: Used for real-time voice recognition via gRPC or WebSocket stream connections.

  12. Authenticate for streaming STT API

    Before connecting to the WebSocket, obtain an access token by sending a POST request to https://openapi.vito.ai/v1/authenticate with your client_id and client_secret as form data.
    data := map[string][]string{
    "client_id":     []string{ClientId},
    "client_secret": []string{ClientSecret},
}
resp, _ := http.PostForm("https://openapi.vito.ai/v1/authenticate", data)