DOCSEARCHbyAlgolia
Submit Docs

RNNoise Documentation

repository·main·Indexed 26 days ago

https://github.com/xiph/rnnoise

A noise suppression library based on a recurrent neural network (RNN) using a hybrid DSP/Deep Learning approach for real-time full-band speech enhancement. Includes tools for training custom models, generating features, and exchanging weights between PyTorch and TensorFlow/Keras. Supports exporting weights to C files and loading models from binary files via the rnnoise_model_from_file() API.

Tokens
1.1K
Snippets
4
Records
8
Agent score
41%

What's inside RNNoise

  1. Exchange weights between PyTorch and TensorFlow/Keras

    main

    Use the weight-exchange utility to move weights between torch and tensorflow.keras modules via an intermediate NumPy format.

    To work with PyTorch weights, import exchange.torch. To work with TensorFlow weights, import exchange.tf.

    Note that import exchange does not automatically import these submodules, allowing you to use the library without having both PyTorch and TensorFlow installed in the same environment simultaneously.

  2. Convert trained models to C files

    main
    After training, convert the resulting .pth files into C source files (rnnoise_data.c and rnnoise_data.h) to be used in the RNNoise build. Use the --quantize flag during conversion.

  3. Generate training features for RNNoise

    main

    To train a custom model, you must first generate feature data by mixing clean speech and noise. Both must be 48 kHz, 16-bit PCM (machine endian).

    Basic feature generation: Use ./dump_features with the speech, background noise, and foreground noise files. The <count> parameter should be at least 10,000 (200,000+ is recommended).

    With Reverberation (RIR): If you have Room Impulse Response (RIR) data (raw 32-bit floating-point, little endian), you can include it using the -rir_list flag.

    Parallel feature generation: To speed up the process, use the provided script script/dump_features_parallel.sh.

    # Basic
./dump_features speech.pcm background_noise.pcm foreground_noise.pcm features.f32 <count>

# With RIR
./dump_features -rir_list rir_list.txt speech.pcm background_noise.pcm foreground_noise.pcm features.f32 <count>

# Parallel execution
script/dump_features_parallel.sh ./dump_features speech.pcm background_noise.pcm foreground_noise.pcm features.f32 <count> rir_list.txt

  4. Train an RNNoise model

    main

    Once the .f32 feature file is generated, start the training process using the Python training script. It is recommended to choose a number of epochs (via --epochs) that results in approximately 75,000 weight updates.

    python3 train_rnnoise.py features.f32 output_directory

  5. Compile and install RNNoise

    main

    To build RNNoise from source, use the standard autotools workflow. Note that autogen.sh will automatically download the required model files from Xiph.Org servers.

    For better performance on x86 architectures, it is recommended to either set -march= in your CFLAGS to an architecture with AVX2 support or add the --enable-x86-rtcd flag to the ./configure script to enable AVX2 or SSE4.1 options.

    ./autogen.sh
./configure
make
# Optionally:
make install

  6. Export weights to C files using exchange.c_export

    main

    The exchange.c_export module provides routines to export weights directly to C files.

    Currently, this functionality is implemented for the following PyTorch layers:

    • torch.nn.GRU
    • torch.nn.Linear
    • torch.nn.Conv1d

    Future updates will allow dump_... functions to accept either a path string or a CWriter instance to determine the export format.

  7. Use the RNNoise command-line demo

    main

    RNNoise provides a simple command-line tool for processing audio.

    Input/Output Requirements:

    • Format: RAW 16-bit (machine endian) mono PCM.
    • Sample Rate: 48 kHz.
    • Warning: The tool operates on RAW PCM files, not WAV files.
    ./examples/rnnoise_demo <noisy speech> <output denoised>

  8. Load models from binary files via API

    main

    RNNoise supports loading models from a binary "machine endian" format using the rnnoise_model_from_file() API.

    Workflow to create a loadable blob:

    1. Build RNNoise with your trained model.
    2. Use the dump_weights_blob executable to output a weights_blob.bin file.
    3. Use rnnoise_model_from_file() to load this binary in your application.

    Critical Constraints:

    • The model object MUST NOT be deleted while the RNNoise state is active.
    • The model file MUST NOT be closed while in use.
    • To reduce build size and rely solely on external loading, add -DUSE_WEIGHTS_FILE to your CFLAGS during compilation.