msedge-tts

repository·main·Indexed 43 days ago

https://github.com/migushthe2nd/msedgetts

An Azure Speech Service module for server-side runtimes that leverages the Microsoft Edge Read Aloud API to convert text into audio streams or files. Version 2.0.7 supports voice customization via ProsodyOptions (rate, pitch, volume), boundary metadata for words and sentences, and custom HTTP Agents for proxy support. It provides methods like toStream(), toFile(), and raw SSML support through rawToStream() and rawToFile().

Tokens
3.9K
Snippets
17
Records
21
Agent score
56%

What's inside msedge-tts

  1. SSML Support and Constraints

    main

    MsEdgeTTS uses a specific SSML template to wrap user input. It does not support full SSML; it only supports the following element types:

    • speak
    • voice
    • prosody

    The generated SSML structure is:

    <speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xmlns:mstts="https://www.w3.org/2001/mstts"
           xml:lang="${this._voiceLang}">
        <voice name="${voiceName}">
            <prosody rate="${rate}" pitch="${pitch}" volume="${volume}">
                ${input}
            </prosody>
        </voice>
    </speak>

    Warning: Because the underlying Read Aloud API requires a user agent matching Microsoft Edge, this library will not work in browsers other than Microsoft Edge. It remains fully functional in server-side runtimes.

  2. Initialize MsEdgeTTS

    main

    Create a new MsEdgeTTS instance to interact with the Microsoft Edge TTS service. You can optionally provide an agent (for proxy support) or enable a built-in logger to see connection and data events in the console.

    Note: The agent option is NOT supported in the browser.

    import { MsEdgeTTS } from 'msedge-tts';
    import { Agent } from 'http';
    
    const client = new MsEdgeTTS({
      agent: new Agent({ /* proxy config */ }),
      enableLogger: true
    });
  3. Adjust voice rate, pitch, and volume

    main

    You can modify the prosody of the speech by passing an options object to toStream. Supported properties include rate and pitch (e.g., "+200Hz").

    import {MsEdgeTTS, OUTPUT_FORMAT} from "msedge-tts";
    
    (async () => {
        const tts = new MsEdgeTTS();
        await tts.setMetadata("en-US-AriaNeural", OUTPUT_FORMAT.WEBM_24KHZ_16BIT_MONO_OPUS);
        const {audioStream} = await tts.toStream("Hi, how are you?", {rate: 0.5, pitch: "+200Hz"});
    })();
  4. Use a custom HTTP Agent for proxy support

    main

    When initializing MsEdgeTTS, you can pass a custom http.Agent (such as https-proxy-agent or socks-proxy-agent) to the constructor to handle requests through a proxy.

    import {MsEdgeTTS, OUTPUT_FORMAT} from "msedge-tts";
    import {SocksProxyAgent} from 'socks-proxy-agent';
    
    (async () => {
        const agent = new SocksProxyAgent("socks://your-name%40gmail.com:abcdef12345124@br41.nordvpn.com")
        const tts = new MsEdgeTTS(agent);
        await tts.setMetadata("en-US-AriaNeural", OUTPUT_FORMAT.WEBM_24KHZ_16BIT_MONO_OPUS);
        const {audioStream} = await tts.toStream("Hi, how are you?");
    })();
  5. Get sentence and word boundaries

    main

    To retrieve metadata regarding word and sentence boundaries, enable them in the setMetadata options.

    When wordBoundaryEnabled or sentenceBoundaryEnabled are set to true, toStream returns a metadataStream and toFile returns a metadataFilePath. The metadata follows this structure:

    {
      "Metadata": [
        {
          "Type": "SentenceBoundary",
          "Data": {
            "Offset": 1000000,
            "Duration": 35875000,
            "text": {
              "Text": "...",
              "Length": 46,
              "BoundaryType": "SentenceBoundary"
            }
          }
        }
      ]
    }
    import {MsEdgeTTS, OUTPUT_FORMAT} from "msedge-tts";
    
    (async () => {
        const tts = new MsEdgeTTS();
        await tts.setMetadata("en-US-AriaNeural", OUTPUT_FORMAT.WEBM_24KHZ_16BIT_MONO_OPUS, {
            wordBoundaryEnabled: true,
            sentenceBoundaryEnabled: true
        });
        // as stream
        const {metadataStream} = await tts.toStream("Hi, how are you doing today hello hello hello?");
        // or as file
        const {metadataFilePath} = await tts.toFile("Hi, how are you?");
    })();
  6. Write text to an audio stream

    main

    Use tts.toStream(text, options) to generate an audio stream from the provided text. This method returns an object containing an audioStream.

    Important: Always escape or sanitize user input before passing it to the TTS engine to prevent SSML injection. You can use a library like xml-escape for this purpose.

    import {MsEdgeTTS, OUTPUT_FORMAT} from "msedge-tts";
    
    const tts = new MsEdgeTTS();
    await tts.setMetadata("en-IE-ConnorNeural", OUTPUT_FORMAT.WEBM_24KHZ_16BIT_MONO_OPUS);
    const {audioStream} = tts.toStream("Hi, how are you?");
    
    audioStream.on("data", (data) => {
        console.log("DATA RECEIVED", data);
        // raw audio file data
    });
    
    audioStream.on("close", () => {
        console.log("STREAM CLOSED");
    });
  7. Write text to an audio file

    main

    Use tts.toFile(directory, text) to save the synthesized audio directly to a file. This method returns a promise that resolves to an object containing audioFilePath.

    import {MsEdgeTTS, OUTPUT_FORMAT} from "msedge-tts";
    
    (async () => {
        const tts = new MsEdgeTTS();
        await tts.setMetadata("en-US-AriaNeural", OUTPUT_FORMAT.WEBM_24KHZ_16BIT_MONO_OPUS);
        const {audioFilePath} = await tts.toFile("./tmpfolder", "Hi, how are you?");  
    })();
  8. Configure prosody with ProsodyOptions

    main

    Use the ProsodyOptions class to adjust the speech characteristics of the generated audio, including pitch, rate (speed), and volume.

    • pitch: Sets the pitch level. You can use the PITCH enum, or provide a string representing a relative frequency in Hz (e.g., "+50Hz"), a relative semitone (e.g., "+2st"), or a relative percentage (e.g., "+50%"). Default is "+0Hz".
    • rate: Sets the speaking rate. You can use the RATE enum, a relative number (e.g., 0.5), or a string with a relative percentage (e.g., "+50%"). Default is 1.0.
    • volume: Sets the volume level. You can use the VOLUME enum, an absolute number (e.g., 0 to 100), or a string with a relative number (e.g., "+50") or relative percentage (e.g., "+50%"). Default is 100.0.
    const options: ProsodyOptions = {
      pitch: "+2st",
      rate: RATE.FAST,
      volume: "+50%"
    };
  9. Configure speech metadata with setMetadata()

    main

    Before synthesizing speech, you must call setMetadata() to configure the voice, output format, and optional boundary metadata. This method initializes the underlying WebSocket connection.

    • voiceName: A string representing the voice (e.g., a neural or standard voice name). Changing this resets the voiceLocale.
    • outputFormat: The desired audio format (from OUTPUT_FORMAT).
    • metadataOptions (optional): Configuration for boundary metadata.

    MetadataOptions properties:

    • voiceLocale: The locale of the voice (e.g., en-US). If not provided, it is inferred from the voiceName using a regex pattern. Changing voiceName resets this.
    • sentenceBoundaryEnabled: Enables sentence boundary metadata (default: false).
    • wordBoundaryEnabled: Enables word boundary metadata (default: false).
    await client.setMetadata(
      'en-US-GuyNeural', 
      'audio-24khz-48kbitrate-mono-mp3', 
      {
        sentenceBoundaryEnabled: true,
        wordBoundaryEnabled: true
      }
    );
  10. Synthesize text to a stream with toStream()

    main

    Synthesizes text in real-time and returns it as a Readable stream. This is useful for low-latency playback or processing.

    • input: The text to synthesize. Can include SSML elements.
    • options (optional): ProsodyOptions to adjust pitch, rate, or volume.

    Returns an object containing audioStream (always present) and metadataStream (present only if boundary metadata was enabled in setMetadata).

    const { audioStream, metadataStream } = client.toStream('Hello world', { pitch: '+2st' });
    
    audioStream.pipe(process.stdout);
    
    if (metadataStream) {
      metadataStream.on('data', (chunk) => {
        console.log('Metadata:', chunk.toString());
      });
    }
  11. Synthesize text to a file with toFile()

    main

    Writes synthesized audio from text to a file in a specified directory. This method uses a basic SSML template internally to wrap your input text.

    • dirPath: The directory where files will be saved.
    • input: The plain text to synthesize.
    • options (optional): ProsodyOptions to adjust pitch, rate, or volume.

    Returns a promise resolving to the file paths for the audio and (if enabled) the metadata.

    const result = await client.toFile('./output', 'Hello world', { rate: '+10%' });
    console.log(result.audioFilePath);
    if (result.metadataFilePath) {
      console.log(result.metadataFilePath);
    }
  12. Use the MsEdgeTTS class

    main
    The primary entry point for the library is the MsEdgeTTS class. It provides methods to convert text to speech using Microsoft Edge's TTS service. You can use it to write audio to a stream, write to a file, or manipulate prosody (rate, pitch, and volume).