sherpa-ncnn

The sherpa-ncnn repository provides local, high-performance implementations of several speech-related functions using the ncnn inference framework. It is designed to run without dependencies on PyTorch or other heavy inference frameworks, depending only on ncnn and system libraries.

Supported Functions:

• Real-time Speech recognition: Streaming speech-to-text.
• Speech synthesis: Text-to-speech (e.g., using VITS models from piper).
• Voice activity detection (VAD): (e.g., using silero-vad).

sherpa-ncnn is an open-source, real-time speech recognition toolkit developed by the Next-gen Kaldi team. It is designed for streaming recognition and operates entirely offline, meaning it does not require an internet connection during the recognition process.

Key features:

• Real-time streaming recognition: Supports continuous speech processing.
• Offline operation: No internet connection required for inference.
• Cross-platform support: Works on Android, iOS, Raspberry Pi, Linux, Windows, macOS, and more.

To use Vulkan-accelerated features on macOS, you must install the Vulkan SDK. Follow these steps to install version 1.3.236.0:

1. Download the installer: https://sdk.lunarg.com/sdk/download/1.3.236.0/mac/vulkansdk-macos-1.3.236.0.dmg
2. Open the .dmg file.
3. Run the InstallVulkan application.
4. Follow the installation wizard:
   • Click Next.
   • Choose an installation directory (e.g., /Users/yourname/software/VulkanSDK/1.3.236.0).
   • Click Next.
   • Click Select All and then Next.
   • Accept the license and click Next.
   • Click Install and enter your system password.
5. Click Finish once complete.

For more detailed information, refer to the official LunarG documentation: https://vulkan.lunarg.com/doc/sdk/latest/mac/getting_started.html

To install the .NET SDK on Linux, use the official dotnet-install.sh script. You can specify a custom installation directory using the --install-dir flag and must update your PATH environment variable to include the installation directory to make the dotnet command available.

wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh
chmod +x dotnet-install.sh
./dotnet-install.sh --help
./dotnet-install.sh --install-dir /star-fj/fangjun/software/dotnet
export PATH=/star-fj/fangjun/software/dotnet:$PATH

# To install the runtime specifically:
./dotnet-install.sh --runtime dotnet --install-dir /star-fj/fangjun/software/dotnet/

To cross-compile for AArch64 (64-bit) architectures, download the gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz toolchain from the Linaro releases page. Extract the archive to your software directory and add the bin directory to your PATH environment variable.

wget https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz

tar xvf gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz -C /ceph-fj/fangjun/software

export PATH=/ceph-fj/fangjun/software/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin:$PATH

After running the service, you can access a web interface for manual testing and experience the speech-to-text functionality at:

http://127.0.0.1:5620

After installation, verify that the dotnet executable is in your path and check the installation details using which and dotnet --info.

which dotnet
dotnet --info

By default, FFmpeg examples are disabled. To enable sherpa-ncnn to use any URL or file input supported by FFmpeg, you must build the project with the SHERPA_NCNN_ENABLE_FFMPEG_EXAMPLES CMake flag set to ON.

Before building, ensure FFmpeg is installed on your system (e.g., via brew install ffmpeg on macOS).

cd sherpa-ncnn
mkdir -p build
cd build
cmake -DSHERPA_NCNN_ENABLE_FFMPEG_EXAMPLES=ON ..
make -j10

If you need to test the functionality immediately or require model files, you can find them at the following locations:

• Streaming speech recognition Android APKs: Available in the GitHub Releases section.
• Pre-trained models: Available at the Models Release Tag.

For detailed installation guides and usage instructions, refer to the official sherpa-ncnn documentation.

To use the sherpa-ncnn WebAssembly module with Node.js, you must first install the project dependencies and download a pre-trained model.

1. Install dependencies: Navigate to the nodejs-examples directory and run npm i.
2. Download a model: Download a model from the official releases page. You will need to extract the model files before running the examples.

cd ./nodejs-examples
npm i

# Example model download and extraction
wget https://github.com/k2-fsa/sherpa-ncnn/releases/download/models/sherpa-ncnn-streaming-zipformer-bilingual-zh-en-2023-02-13.tar.bz2
tar xvf sherpa-ncnn-streaming-zipformer-bilingual-zh-en-2023-02-13.tar.bz2
rm sherpa-ncnn-streaming-zipformer-bilingual-zh-en-2023-02-13.tar.bz2

To cross-compile for ARM (32-bit) architectures, download the gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz toolchain from the ARM Developer website. After downloading, extract the archive and add the bin directory to your PATH environment variable.

mkdir /ceph-fj/fangjun/software
cd /ceph-fj/fangjun/software
tar xvf /path/to/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz

export PATH=/ceph-fj/fangjun/software/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf/bin:$PATH

Sherpa-ncnn provides Node.js examples for performing real-time speech-to-text locally. All processing is done on-device without internet access. For specific implementation patterns and usage examples, refer to the nodejs-examples directory in the repository.

https://github.com/k2-fsa/sherpa-ncnn/tree/master/nodejs-examples