NISQA

repository·master·Indexed 21 days ago

A deep learning framework for assessing speech quality and naturalness. NISQA can predict multidimensional quality metrics for transmitted speech or estimate the naturalness of synthetic speech (TTS). The framework supports prediction via run_predict.py, finetuning for transfer learning, training new model architectures using YAML configurations, and model evaluation using run_evaluate.py.

Tokens: 1.2K
Snippets: 5
Records: 6
Agent score: 26%

What's inside NISQA

Install NISQA via Conda
master
To install the necessary requirements, use Anaconda to create a new environment from the provided env.yml file. This will create an environment named nisqa.
```
conda env create -f env.yml
conda activate nisqa
```
https://github.com/gabrielmittag/nisqa/blob/master/README.md
Train a new NISQA model architecture
master
NISQA allows training new models by defining a structure in a YAML configuration. The architecture typically consists of:
1. Framewise model: CNN or Feedforward.
2. Time-Dependency model: Self-Attention or LSTM.
3. Pooling: Average, Max, Attention, or Last-Step-Pooling.
You can skip stages (e.g., training an LSTM without a CNN) or add multiple time-dependency stages. For full-reference (double-ended) speech quality prediction, use the train_nisqa_double_ended.yaml configuration.
```
# Example: Training with CNN, Self-Attention, and Attention-Pooling
python run_train.py --yaml config/train_nisqa_cnn_sa_ap.yaml
```
https://github.com/gabrielmittag/nisqa/blob/master/README.md
Evaluate trained models
master
Use run_evaluate.py to evaluate model performance or perform a conformance test.
Setup: You must update the paths and options inside the run_evaluate.py script before execution. If using the NISQA Corpus, update data_dir and output_dir within the script.
Metrics:
- Pearson's Correlation
- RMSE
- RMSE after first-order polynomial mapping
- Per-condition results (if a CSV with per-condition labels is provided)
```
python run_evaluate.py
```
https://github.com/gabrielmittag/nisqa/blob/master/README.md

Predict speech quality using NISQA

master

NISQA supports three prediction modes via the run_predict.py script: predicting a single file, a directory of files, or a list of files from a CSV.

Model Selection:

For transmitted speech (overall quality + dimensions like Noisiness, Coloration, etc.): Use weights/nisqa.tar.
For synthesized speech (Naturalness): Use weights/nisqa_tts.tar.

Optimization: Use --num_workers and --bs (batch size) to speed up the Pytorch Dataloader. For stereo files, use --ms_channel to select the specific audio channel.

# Predict a single .wav file
python run_predict.py --mode predict_file --pretrained_model weights/nisqa.tar --deg /path/to/wav/file.wav --output_dir /path/to/dir/with/results

# Predict all .wav files in a folder
python run_predict.py --mode predict_dir --pretrained_model weights/nisqa.tar --data_dir /path/to/folder/with/wavs --num_workers 0 --bs 10 --output_dir /path/to/dir/with/results

# Predict all .wav files listed in a CSV
python run_predict.py --mode predict_csv --pretrained_model weights/nisqa.tar --csv_file files.csv --csv_deg column_name_of_filepaths --num_workers 0 --bs 10 --output_dir /path/to/dir/with/results

Finetune NISQA for transfer learning
master
You can finetune existing model weights on new datasets using a YAML configuration file.
CSV Requirements: Your dataset CSV must contain:
- db: Dataset names.
- filepath_deg: Path to the degraded WAV file (absolute or relative to data_dir).
- mos: Target labels.
YAML Configuration Keys:
- data_dir: Path to the main folder containing the CSV and datasets.
- output_dir: Path for saved weights and results.
- pretrained_model: nisqa_mos_only.tar (for natural speech) or nisqa_tts.tar (for synthesized speech).
- csv_file: Name of the CSV file.
- csv_deg: Column name for filepaths.
- csv_mos_train / csv_mos_val: Column names for target values.
- csv_db_train / csv_db_val: Column names for dataset names used for training/validation.
```
python run_train.py --yaml config/finetune_nisqa.yaml
```
https://github.com/gabrielmittag/nisqa/blob/master/README.md

Reference: Available NISQA Model Weights

master

Choose the appropriate weight file based on your target domain:

Model	Prediction Output	Domain	Filename
NISQA (v2.0)	Overall Quality, Noisiness, Coloration, Discontinuity, Loudness	Transmitted Speech	`nisqa.tar`
NISQA (v2.0) mos only	Overall Quality only (for finetuning/transfer learning)	Transmitted Speech	`nisqa_mos_only.tar`
NISQA-TTS (v1.0)	Naturalness	Synthesized Speech	`nisqa_tts.tar`