whisper.unity Documentation

The repository includes ggml-tiny.bin by default, which is fast but lower quality. To use larger or specialized models (like English-only weights):

1. Download model weights in GGML format from the whisper.cpp Hugging Face repository.
2. Place the downloaded .bin files into your Unity project's StreamingAssets folder.

If you need to rebuild the underlying C++ libraries for a specific platform, follow these steps. Note that this project is specifically compatible with whisper.cpp version v1.7.5.

Prerequisites

1. Clone the whisper.cpp repository.
2. Checkout tag v1.7.5.

Build Commands

Run these commands from within the whisper.unity folder:

Windows:

.uild_cpp.bat path\to\whisper

MacOS (includes iOS and Android):

sh build_cpp.sh path/to/whisper all path/to/ndk/android.toolchain.cmake

Linux:

sh build_cpp_linux.sh path/to/whisper

Upon a successful build, the compiled libraries will be automatically updated in the package's Plugins folder.

To improve inference performance, you can enable GPU acceleration. The project uses Vulkan for Windows and Linux, and Metal for macOS, iOS, and visionOS.

How to activate:

1. Locate the WhisperManager component in your Unity scene.
2. Enable the Use GPU toggle.

Important Notes:

• Fallback: If the hardware is unsupported, the system will automatically fall back to CPU inference.
• Metal Requirements: Metal support requires Apple GPUs from the Apple M1 chip family or newer. Older hardware will fall back to CPU.
• CUDA: CUDA is no longer supported; use Vulkan instead for Windows/Linux GPU acceleration.

You can add whisper.unity to your existing Unity project using the Unity Package Manager. This is the recommended way to integrate the bindings without cloning the entire repository.

1. Open your Unity project.
2. Open the Package Manager.
3. Click the + icon and select Add package from git URL....
4. Enter the following URL: https://github.com/Macoron/whisper.unity.git?path=/Packages/com.whisper.unity