Whisper.net

Whisper.net supports multiple runtimes in a single project. The NativeLibraryLoader automatically selects the best available runtime based on the platform and hardware.

By default, the priority order is:

1. Whisper.net.Runtime.Cuda (CUDA 13)
2. Whisper.net.Runtime.Cuda12 (CUDA 12)
3. Whisper.net.Runtime.Vulkan (Windows x64)
4. Whisper.net.Runtime.CoreML (Apple)
5. Whisper.net.Runtime.OpenVino (Intel)
6. Whisper.net.Runtime (CPU)
7. Whisper.net.Runtime.NoAvx (CPU without AVX)

The loader probes CUDA runtimes and validates drivers via cudaRuntimeGetVersion to allow transparent fallback from CUDA 13 to CUDA 12.

To generate a production-ready version of the website, run the build command. This will create a dist/ folder containing the static assets required for deployment.

npm run build

The Whisper.net presentation website is a static single-page site built with React and Vite. To run it locally for development, ensure you have Node.js (v18 or later) installed, then navigate to the website directory to install dependencies and start the dev server.

cd website
npm install
npm run dev

To install Whisper.net with all available runtimes (including CUDA 12 and 13), use the Whisper.net.AllRuntimes package.

Whisper.net is the core package containing functionality but requires a runtime package to be installed separately to perform inference.

# Using Package Manager Console
Install-Package Whisper.net.AllRuntimes

<!-- Using .csproj reference -->
<PackageReference Include="Whisper.net.AllRuntimes" Version="1.9.1" />

If you do not want to include all runtimes, you can install the core package and a specific runtime package individually. For example, to use the default CPU runtime:

1. Install Whisper.net.
2. Install Whisper.net.Runtime.

<PackageReference Include="Whisper.net" Version="1.9.1" />
<PackageReference Include="Whisper.net.Runtime" Version="1.9.1" />

Whisper.net uses Ggml models. You can use the built-in WhisperGgmlDownloader to fetch models from Hugging Face. You can also set the HF_TOKEN environment variable to avoid rate limiting.

var modelName = "ggml-base.bin";
if (!File.Exists(modelName))
{
    using var modelStream = await WhisperGgmlDownloader.Default.GetGgmlModelAsync(GgmlType.Base);
    using var fileWriter = File.OpenWrite(modelName);
    await modelStream.CopyToAsync(fileWriter);
}

The WhisperVadFactory uses Silero VAD models. These can be downloaded via the WhisperGgmlDownloader.

var vadModelName = "ggml-silero-v6.2.0.bin";
if (!File.Exists(vadModelName))
{
    using var modelStream = await WhisperGgmlDownloader.Default.GetGgmlSileroVadModelAsync();
    using var fileWriter = File.OpenWrite(vadModelName);
    await modelStream.CopyToAsync(fileWriter);
}

using var vadFactory = WhisperVadFactory.FromPath(vadModelName);

To run the test suites locally, refer to tests/README.md for specific requirements regarding .NET SDKs and environment variables like HF_TOKEN.

Offline Testing

You can run tests without network access by pre-downloading the required ggml models and setting the WHISPER_TEST_MODEL_PATH environment variable to point to the local model directory.

MAUI Tests

Tests for MAUI require the Dotnet XHarness CLI to manage emulators and simulators.

# Example: Running tests offline
export WHISPER_TEST_MODEL_PATH=/path/to/your/local/models
dotnet test

To use Whisper.net, create a WhisperFactory from a model path, build a processor using the CreateBuilder() method, and then process an audio file stream using ProcessAsync.

using var whisperFactory = WhisperFactory.FromPath("ggml-base.bin");

using var processor = whisperFactory.CreateBuilder()
    .WithLanguage("auto")
    .Build();

using var fileStream = File.OpenRead(wavFileName);

await foreach (var result in processor.ProcessAsync(fileStream))
{
    Console.WriteLine($"{result.Start}->{result.End}: {result.Text}");
}

You can change the automatic selection order or force a specific runtime by setting the RuntimeLibraryOrder on RuntimeOptions using RuntimeLibrary enums.

RuntimeOptions.RuntimeLibraryOrder =
[
    RuntimeLibrary.CoreML,
    RuntimeLibrary.OpenVino,
    RuntimeLibrary.Cuda,
    RuntimeLibrary.Cuda12,
    RuntimeLibrary.Cpu
];

If your models are not in a standard file path, you can load them using WhisperFactory.FromStream, WhisperFactory.FromBuffer, or by implementing the IWhisperModelLoader interface.

using var whisperModelStream = await GetWhisperModelStreamAsync();
using var whisperFactory = WhisperFactory.FromStream(whisperModelStream);

using var vadModelStream = await GetVadModelStreamAsync();
using var vadFactory = WhisperVadFactory.FromStream(vadModelStream);

Whisper.net provides multiple runtimes for different hardware acceleration and platforms. Choose the one that matches your target environment and hardware capabilities.

### Available Runtimes:
- `Whisper.net.Runtime`: Default CPU runtime (requires AVX, AVX2, FMA, F16C).
- `Whisper.net.Runtime.NoAvx`: CPU runtime for older CPUs lacking AVX support.
- `Whisper.net.Runtime.Cuda`: NVidia GPU support (built with CUDA 13).
- `Whisper.net.Runtime.Cuda12`: NVidia GPU support (built with CUDA 12).
- `Whisper.net.Runtime.CoreML`: Apple CoreML support (macOS, iOS, MacCatalyst).
- `Whisper.net.Runtime.OpenVino`: Intel OpenVino support.
- `Whisper.net.Runtime.Vulkan`: Vulkan support (Windows x64).