DOCSEARCHbyAlgolia
Submit Docs

Kokoro Text-to-Speech

repository·main·Indexed 27 days ago

https://github.com/hexgrad/kokoro

A lightweight, high-performance 82M parameter open-weight Text-to-Speech (TTS) model. It provides a Python library featuring KPipeline and KModel for core inference, as well as a JavaScript library (kokoro-js) for running TTS 100% locally in the browser via WebGPU or WASM. Supports multiple languages including American and British English, Spanish, French, Hindi, Italian, Brazilian Portuguese, Japanese, and Mandarin Chinese.

Tokens
5K
Snippets
12
Records
32
Agent score
91%

What's inside Kokoro

  1. Set up the Kokoro Text-to-Speech demo application

    main

    To run the React + Vite demo application which runs the Kokoro model 100% locally in the browser using kokoro-js and @huggingface/transformers, follow these steps in order:

    1. Clone the repository
    2. Build the core dependencies in the kokoro.js directory.
    3. Install demo dependencies in the demo directory.
    4. Start the development server.

    The application will be available at http://localhost:5173.

    # 1. Clone the Repository
git clone https://github.com/hexgrad/kokoro.git

# 2. Build the Dependencies
cd kokoro/kokoro.js
npm i
npm run build

# 3. Setup the Demo Project
cd ../demo
npm i

# 4. Start the Development Server
npm run dev

  3. Enable GPU acceleration on macOS Apple Silicon

    main

    To enable GPU acceleration on M1/M2/M3/M4 Mac devices, set the PYTORCH_ENABLE_MPS_FALLBACK=1 environment variable when running your script.

    PYTORCH_ENABLE_MPS_FALLBACK=1 python run-your-kokoro-script.py

  4. Install kokoro and dependencies

    main

    Install the kokoro library and soundfile via pip. You must also install espeak-ng on your system, as it is used for English OOD fallback and certain non-English languages.

    Linux (Ubuntu/Debian):

    apt-get -qq -y install espeak-ng

    Windows:

    1. Download the appropriate *.msi file from the espeak-ng releases.
    2. Run the installer.

    macOS: Follow standard installation procedures for espeak-ng.

    pip install -q kokoro>=0.9.4 soundfile

  5. Configure Conda environment for Kokoro

    main

    If you encounter dependency issues, use the following environment.yml configuration. Note that libstdcxx~=12.4.0 is often required to load espeak correctly.

    name: kokoro
channels:
  - defaults
dependencies:
  - python==3.9       
  - libstdcxx~=12.4.0
  - pip:
      - kokoro>=0.3.1
      - soundfile
      - misaki[en]

  6. Stream audio output with TextSplitterStream

    main

    For real-time applications (like consuming LLM tokens), use TextSplitterStream and tts.stream(splitter) to process text incrementally.

    1. Create a TextSplitterStream instance.
    2. Initialize the stream with tts.stream(splitter).
    3. Iterate over the stream using for await...of to receive chunks containing { text, phonemes, audio }.
    4. Push text to the splitter using splitter.push(token).
    5. Use splitter.close() to signal the end of input, or splitter.flush() to process remaining text without closing the stream.
    import { KokoroTTS, TextSplitterStream } from "kokoro-js";

const model_id = "onnx-community/Kokoro-82M-v1.0-ONNX";
const tts = await KokoroTTS.from_pretrained(model_id, {
  dtype: "fp32",
});

const splitter = new TextSplitterStream();
const stream = tts.stream(splitter);

(async () => {
  let i = 0;
  for await (const { text, phonemes, audio } of stream) {
    console.log({ text, phonemes });
    audio.save(`audio-${i++}.wav`);
  }
})();

const text = "Kokoro is an open-weight TTS model...";
const tokens = text.match(/\s*\S+/g);
for (const token of tokens) {
  splitter.push(token);
  await new Promise((resolve) => setTimeout(resolve, 10));
}

splitter.close();

  7. Generate audio with KPipeline

    main

    Call the KPipeline instance as a function to generate audio. It returns a generator that yields tuples containing graphemes, phonemes, and the audio data.

    Arguments:

    • text: The input string to synthesize.
    • voice: The name of the voice to use (e.g., 'af_heart'). You can also pass a loaded torch voice tensor.
    • speed: The playback speed (default is 1).
    • split_pattern: A regex pattern used to split text (e.g., r'\n+').

    Yields: Each iteration returns (gs, ps, audio) where:

    • gs: Graphemes/text
    • ps: Phonemes
    • audio: The generated audio data (numpy array/tensor)

    Example:

    from kokoro import KPipeline
import soundfile as sf

pipeline = KPipeline(lang_code='a')
text = "Hello world"

generator = pipeline(text, voice='af_heart', speed=1, split_pattern=r'\n+')

for i, (gs, ps, audio) in enumerate(generator):
    print(f"Segment {i}: {gs}")
    sf.write(f'{i}.wav', audio, 24000)
    generator = pipeline(
    text, voice='af_heart', # <= change voice here
    speed=1, split_pattern=r'\n+'
)

for i, (gs, ps, audio) in enumerate(generator):
    print(i)  # i => index
    print(gs) # gs => graphemes/text
    print(ps) # ps => phonemes
    sf.write(f'{i}.wav', audio, 24000) # save each audio file

  8. Generate speech with KokoroTTS.from_pretrained()

    main

    Initialize the TTS engine using KokoroTTS.from_pretrained(model_id, options). You can specify the dtype (quantization) and the device (runtime environment).

    Options:

    • dtype: "fp32", "fp16", "q8", "q4", "q4f16"
    • device: "wasm", "webgpu" (for web), or "cpu" (for Node.js). If using "webgpu", it is recommended to use dtype="fp32".
    import { KokoroTTS } from "kokoro-js";

const model_id = "onnx-community/Kokoro-82M-v1.0-ONNX";
const tts = await KokoroTTS.from_pretrained(model_id, {
  dtype: "q8", // Options: "fp32", "fp16", "q8", "q4", "q4f16"
  device: "wasm", // Options: "wasm", "webgpu" (web) or "cpu" (node).
});

const text = "Life is like a box of chocolates. You never know what you're gonna get.";
const audio = await tts.generate(text, {
  // Use `tts.list_voices()` to list all available voices
  voice: "af_heart",
});
audio.save("audio.wav");

  9. Initialize a KPipeline for Text-to-Speech

    main

    Use KPipeline to initialize the TTS engine. You must provide a lang_code that matches the voice you intend to use.

    Supported lang_code values:

    • a: American English
    • b: British English
    • e: Spanish (es)
    • f: French (fr-fr)
    • h: Hindi (hi)
    • i: Italian (it)
    • j: Japanese (requires pip install misaki[ja])
    • p: Brazilian Portuguese (pt-br)
    • z: Mandarin Chinese (requires pip install misaki[zh])

    Note: For Japanese or Chinese, you must also install the corresponding misaki extension.

    from kokoro import KPipeline

# Example for American English
pipeline = KPipeline(lang_code='a')

  10. Reference available voices

    main

    Voices are identified by a string ID passed to the voice option in tts.generate(). You can retrieve the full list of available voices using tts.list_voices().

    Common voice prefixes:

    • af_: American English Female
    • am_: American English Male
    • bf_: British English Female
    • bm_: British English Male

    Detailed samples can be found on the Hugging Face model card.

  11. Initialize KokoroTTS from a pretrained model

    main

    Use KokoroTTS.from_pretrained(model_id, options) to load a model and its tokenizer from the Hugging Face Hub. This is the recommended way to instantiate the class.

    Options:

    • dtype: The data type to use. Options: `

  12. Troubleshoot Kokoro CLI issues

    main

    If you encounter issues running the CLI, check the following:

    • pip not installed: Run uv pip install pip.
    • espeak not installed: The CLI requires espeak-ng for English OOD fallback and some non-English languages. Install it via apt-get install espeak-ng.
    • Output format: The CLI expects the output file name to end with .wav. If it does not, a warning will be issued.