DOCSEARCHbyAlgolia
Submit Docs

nodejs-whisper

repository·main·Indexed 42 days ago

https://github.com/chetanxpro/nodejs-whisper

Node.js bindings for OpenAI's Whisper model using whisper.cpp, optimized for CPU and Apple Silicon. Version 0.3.1 provides the `nodewhisper` function for audio transcription with support for automatic audio conversion, multiple output formats (Text, VTT, SRT, CSV, JSON), and Silero Voice Activity Detection (VAD).

Tokens
4.7K
Snippets
15
Records
17
Agent score
49%

What's inside nodejs-whisper

  1. Enable Voice Activity Detection (VAD)

    main

    VAD detects speech before transcription, which reduces processing time for long recordings containing silence. The recommended Silero model is small (< 1 MB) and can be automatically downloaded.

    Automatic Setup: Provide autoDownloadVadModelName in the main options object (e.g., 'silero-v6.2.0').

    Manual Setup: Set whisperOptions.vad to true and provide the path via whisperOptions.vadModelPath.

    await nodewhisper(filePath, {
	modelName: 'tiny.en',
	autoDownloadModelName: 'tiny.en',
	autoDownloadVadModelName: 'silero-v6.2.0',
	whisperOptions: {
		vadThreshold: 0.5,
		vadMinSilenceDurationMs: 100,
	},
})

  2. Install nodejs-whisper on Linux

    main

    To install nodejs-whisper on a Linux system, you must first install the necessary build tools (make, gcc, etc.) and then install the package via npm. Finally, you must download the Whisper model files.

    1. Install build essentials:
      sudo apt update
sudo apt install build-essential
    2. Install the package:
      npm i nodejs-whisper
    3. Download the Whisper model:
      npx nodejs-whisper download
    sudo apt update
apt install build-essential
npm i nodejs-whisper
npx nodejs-whisper download

  3. Install nodejs-whisper on Windows

    main

    To install nodejs-whisper on Windows, ensure you have make tools available in your system PATH (via MinGW-w64 or MSYS2).

    1. Install MinGW-w64 or MSYS2 (MSYS2 is recommended).
    2. Install the package:
      npm i nodejs-whisper
    3. Download the Whisper model:
      npx nodejs-whisper download
    npm i nodejs-whisper
npx nodejs-whisper download

  5. Enable Voice Activity Detection (VAD) automatically

    main

    To use Voice Activity Detection (VAD) to improve transcription, provide the autoDownloadVadModelName property in your IOptions. This triggers an automatic download of the VAD model and configures the internal whisperOptions to enable vad: true and set the correct vadModelPath.

    import { nodewhisper } from 'nodejs-whisper';
import { VadModelName } from 'nodejs-whisper';

await nodewhisper('audio.mp3', {
  modelName: 'base',
  autoDownloadVadModelName: 'silero' as VadModelName // Example VAD model name
});

  6. Use nodewhisper() for transcription

    main

    The primary function for transcribing audio is nodewhisper(filePath, options). It automatically converts audio to 16000 Hz WAV format to support the Whisper model.

    Key Options:

    • modelName: The name of the downloaded GGML model (e.g., 'base.en').
    • modelRootPath: (Optional) The directory containing the GGML model files.
    • autoDownloadModelName: (Optional) Automatically downloads a model if not present.
    • autoDownloadVadModelName: (Optional) Downloads and enables a Silero VAD model.
    • whisperOptions: Configuration for output formats and VAD behavior.

    Logger Behavior:

    • logger.log: Receives transcript output.
    • logger.debug: Receives whisper.cpp initialization and progress details.
    • logger.error: Receives command failures.
    import path from 'path'
import { nodewhisper } from 'nodejs-whisper'

const filePath = path.resolve(__dirname, 'YourAudioFileName')

await nodewhisper(filePath, {
	modelName: 'base.en',
	modelRootPath: '/path/to/whisper/models',
	autoDownloadModelName: 'base.en',
	autoDownloadVadModelName: 'silero-v6.2.0',
	removeWavFileAfterTranscription: false,
	withCuda: false,
	logger: console,
	whisperOptions: {
		outputInSrt: true,
		translateToEnglish: false,
		wordTimestamps: false,
		splitOnWord: true,
	},
})

  7. Configure Whisper transcription options

    main

    The whisperOptions object allows fine-grained control over the transcription process.

    Output Formats:

    • outputInCsv, outputInJson, outputInJsonFull, outputInLrc, outputInSrt, outputInText, outputInVtt, outputInWords (wts).

    Transcription Settings:

    • translateToEnglish: Translate from source language to English.
    • wordTimestamps: Enable word-level timestamps.
    • splitOnWord: Split on word rather than on token.
    • timestamps_length: Amount of dialogue per timestamp pair.

    GPU/Hardware:

    • noGpu: Disable GPU inference.

    Voice Activity Detection (VAD) Settings:

    • vad: Enable VAD (requires vadModelPath).
    • vadModelPath: Path to a custom VAD model.
    • vadThreshold: Speech detection probability threshold.
    • vadMinSpeechDurationMs: Discard shorter speech segments.
    • vadMinSilenceDurationMs: Silence required to split segments.
    • vadMaxSpeechDurationS: Split speech segments longer than this.
    • vadSpeechPadMs: Padding around detected speech.
    • vadSamplesOverlap: Overlap between speech segments in seconds.
     interface WhisperOptions {
	outputInCsv?: boolean
	outputInJson?: boolean
	outputInJsonFull?: boolean
	outputInLrc?: boolean
	outputInSrt?: boolean
	outputInText?: boolean
	outputInVtt?: boolean
	outputInWords?: boolean
	translateToEnglish?: boolean
	timestamps_length?: number
	wordTimestamps?: boolean
	splitOnWord?: boolean
	noGpu?: boolean
	vad?: boolean
	vadModelPath?: string
	vadThreshold?: number
	vadMinSpeechDurationMs?: number
	vadMinSilenceDurationMs?: number
	vadMaxSpeechDurationS?: number
	vadSpeechPadMs?: number
	vadSamplesOverlap?: number
}

  8. Available Whisper Models

    main

    The following model names are supported for modelName or autoDownloadModelName:

    • tiny, tiny.en, tiny-q5_1, tiny.en-q5_1, tiny-q8_0
    • base, base.en, base-q5_1, base.en-q5_1, base-q8_0
    • small, small.en, small.en-tdrz, small-q5_1, small.en-q5_1, small-q8_0
    • medium, medium.en, medium-q5_0, medium.en-q5_0, medium-q8_0
    • large-v1, large-v2, large-v2-q5_0, large-v2-q8_0, large-v3, large-v3-q5_0, large-v3-turbo, large-v3-turbo-q5_0, large-v3-turbo-q8_0, large (alias for large-v3)

  9. Transcribe audio with nodewhisper()

    main

    The nodewhisper function is the primary entrypoint for transcribing audio files using Whisper. It handles model downloading (if configured), audio format conversion to WAV, command construction, and execution of the underlying C++ Whisper process.

    It returns the transcription text as a string. If the transcription fails or produces no output, it throws an error.

    import { nodewhisper } from 'nodejs-whisper';

const transcript = await nodewhisper('path/to/audio.mp3', {
  modelName: 'base',
  whisperOptions: {
    // ... other WhisperOptions
  }
});

console.log(transcript);

  10. Validate Whisper and VAD model names

    main

    To ensure a string is a valid model identifier before passing it to the library, use the isModelName and isVadModelName type guards. This prevents errors when attempting to download or load non-existent models.

    import { isModelName, isVadModelName } from 'nodejs-whisper';

const myModel = 'tiny.en';
if (isModelName(myModel)) {
  // myModel is typed as ModelName
}

const myVad = 'silero-v6.2.0';
if (isVadModelName(myVad)) {
  // myVad is typed as VadModelName
}

  11. Configure Whisper transcription with WhisperOptions

    main

    The WhisperOptions interface defines the configuration for the transcription process. It allows you to control output formats, language settings, hardware usage, and Voice Activity Detection (VAD) parameters.

    Output Formats

    You can specify one or more output formats using boolean flags:

    • outputInCsv: Output as CSV.
    • outputInJson: Output as JSON.
    • outputInJsonFull: Output as full JSON (including more metadata).
    • outputInLrc: Output as LRC (lyrics).
    • outputInSrt: Output as SRT (subtitles).
    • outputInText: Output as plain text.
    • outputInVtt: Output as VTT (web video subtitles).
    • outputInWords: Output word-level data.

    Transcription Settings

    • translateToEnglish: Boolean to translate the transcription to English.
    • language: The source language string (e.g., 'en', 'fr').
    • timestamps_length: Length of timestamps.
    • wordTimestamps: Enable word-level timestamps.
    • splitOnWord: Split transcription on word boundaries.

    Hardware and VAD

    • noGpu: If true, disables GPU acceleration.
    • vad: Enables Voice Activity Detection.
    • vadModelPath: Path to the VAD model file.
    • vadThreshold: Sensitivity threshold for VAD.
    • vadMinSpeechDurationMs: Minimum speech duration in milliseconds.
    • vadMinSilenceDurationMs: Minimum silence duration in milliseconds.
    • vadMaxSpeechDurationS: Maximum speech duration in seconds.
    • vadSpeechPadMs: Padding for speech segments in milliseconds.
    • vadSamplesOverlap: Number of samples to overlap in VAD segments.
    const options: WhisperOptions = {
  outputInSrt: true,
  language: 'en',
  translateToEnglish: false,
  vad: true,
  vadThreshold: 0.5
};

  12. Use the nodewhisper function for transcription

    main

    The primary entrypoint for the library is the nodewhisper function. It takes a path to an audio file and an options object, returning a Promise<string> that resolves to the transcription result.

    import { nodewhisper } from 'nodejs-whisper';

const transcription = await nodewhisper('path/to/audio.wav', {
  modelName: 'base',
  whisperOptions: {
    outputInText: true
  }
});