DOCSEARCHbyAlgolia
Submit Docs

Smart Turn Documentation

repository·main·Indexed 23 days ago

https://github.com/pipecat-ai/smart-turn

An open model designed to detect whether a speaker's turn is complete or incomplete for voice-based AI applications. Includes guidelines for preparing audio training data, labeling speech samples, dataset organization, and contribution requirements.

Tokens
606
Snippets
0
Records
4
Agent score
31%

What's inside Smart Turn

  1. Label speech samples as 'complete' or 'incomplete'

    main

    Smart Turn requires a 50:50 split between 'complete' and 'incomplete' samples to ensure unbiased training. Samples must be grouped by language and label type.

    Complete Samples

    A sample is "complete" if it represents a finished thought where a natural response can follow immediately. *Examples: "How tall is the Eiffel Tower?", "Yes.", "My PIN code is five, three, two, seven."

    Incomplete Samples

    A sample is "incomplete" if the speaker is likely to continue talking. These samples must end in one of the following:

    • A filler word (e.g., "um", "er", "well")
    • A connective word (e.g., "and", "but", "because")
    • Prosody suggesting thinking (e.g., drawing out the last syllable or a hanging pitch contour)

    CRITICAL RULE: Incomplete samples must not be cut off in the middle of a word.

    • ❌ Incorrect: "How tall is the Eiff"
    • ✅ Correct: "How tall is the Eiffel..."

  3. Organize and submit Smart Turn datasets

    main

    You may use any naming scheme or directory structure. A common pattern is using a unique UUID for filenames and directory names for labels: eng/incomplete/b3799254-8d6c-11f0-a90e-e7e92780240b.flac

    Alternatively, you can provide a separate metadata file like JSONL.

    Submission Process

    There are no strict submission requirements for the transport method. You can use:

    • Shared cloud storage (e.g., S3, Google Drive)
    • A HuggingFace repository

    To initiate a submission, contact the maintainers via GitHub issues.

  4. Prepare audio files for Smart Turn training data

    main

    When preparing audio samples for Smart Turn, follow these technical specifications:

    • Format: Use FLAC (preferred). Avoid lossy formats like MP3 or Opus.
    • Channels: Use Mono audio.
    • Bit Depth: Use 16 bits.
    • Sample Rate: 16kHz is the model's working rate, but higher sample rates are acceptable.
    • Length: Each file must contain exactly one speech sample and must be no longer than 16 seconds.
    • Silence: To align with the VAD (Voice Activity Detection) model, each sample should ideally end with approximately 200ms of silence (more is fine).