ViSQOL (Virtual Speech Quality Objective Listener)

If ViSQOL's default MOS predictions are inaccurate for your specific use case, you can train a custom SVR model using the libsvm codebase. Note that SVR is currently only supported for audio mode.

Training Procedure

1. Data Collection: Gather pairs of audio files (48kHz for audio mode) along with their corresponding subjective test scores.
2. File Preparation:
   • Create a CSV file listing the file pairs for comparison (compatible with --batch_input_csv).
   • Create a second CSV containing the MOS-LQS (mean subjective scores) in a column named moslqs, corresponding to the rows in the first CSV.
3. Code Modification: Modify src/include/sim_results_writer.h to set output_fvnsim=true and output_moslqo=false.
4. Batch Processing: Run ViSQOLAudio in batch mode using the --batch_input_csv and --output_csv flags.
5. Model Generation:
   • Run the script scripts/make_svm_train_file.py on your output CSV.
   • Perform a grid search to optimize SVM parameters (refer to the script's documentation for help).
6. Deployment: Pass your new model to ViSQOL in audio mode using the --similarity_to_quality_model flag.

ViSQOL can be used as a CLI tool to compare audio files. Note: Input signals must be in WAV format.

Single File Comparison

Use --reference_file and --degraded_file to compare two specific WAV files.

Batch Processing

Provide a CSV file via --batch_input_csv containing pairs of files. The format should be:

reference,degraded
ref1.wav,deg1.wav
ref2.wav,deg2.wav

When using batch mode, the individual file flags are ignored.

Output Formats

• Results: Use --results_csv to save scores to a CSV file (format: reference,degraded,moslqo).
• Debug Info: Use --output_debug to save detailed comparison data in JSON format. This file is appended to if it already exists.
• Console Output: Use --verbose to print file paths and MOS-LQO values to the console, along with per-patch and per-frequency band similarity scores.

# Compare two files and output similarity to console (Linux/Mac)
./bazel-bin/visqol --reference_file ref1.wav --degraded_file deg1.wav --verbose

# Batch processing with results and debug JSON (Linux/Mac)
./bazel-bin/visqol --batch_input_csv input.csv --results_csv results.csv --output_debug debug.json

# Compare using scaled speech mode (Linux/Mac)
./bazel-bin/visqol --reference_file ref1.wav --degraded_file deg1.wav --use_speech_mode --verbose

The ViSQOL application can be executed as a command-line tool to compare signal pairs (reference vs. degraded). The execution flow involves parsing command-line arguments, initializing the VisqolManager with specific model and mode settings, and then iterating through file pairs to run the comparison and write results.

Key components used in the CLI:

• Visqol::VisqolCommandLineParser: Handles argument parsing and builds file paths for comparison.
• Visqol::VisqolManager: The core engine that must be initialized before running comparisons.
• Visqol::SimilarityResultsWriter: Handles outputting results to CSV or debug paths.

Note: If the manager encounters an absl::StatusCode::kAborted error, it typically indicates that the manager was not properly initialized, and the process should terminate.

// Conceptual flow of the ViSQOL CLI execution

// 1. Parse arguments
auto parse_statusor = Visqol::VisqolCommandLineParser::Parse(argc, argv);
Visqol::CommandLineArgs cmd_args = parse_statusor.value();

// 2. Build file pairs
auto files_to_compare = Visqol::VisqolCommandLineParser::BuildFilePairPaths(cmd_args);

// 3. Initialize Manager
Visqol::VisqolManager visqol;
visqol.Init(
    cmd_args.similarity_to_quality_mapper_model, 
    cmd_args.use_speech_mode, 
    cmd_args.use_unscaled_speech_mos_mapping, 
    cmd_args.search_window_radius, 
    cmd_args.use_lattice_model, 
    cmd_args.disable_global_alignment, 
    cmd_args.disable_realignment
);

// 4. Run and Write Results
for (const auto& signal_pair : files_to_compare) {
    auto status_or = visqol.Run(signal_pair.reference, signal_pair.degraded);
    if (status_or.ok()) {
        Visqol::SimilarityResultsWriter::Write(
            cmd_args.verbose, 
            cmd_args.results_output_csv, 
            cmd_args.debug_output_path, 
            status_or.value(),
            cmd_args.use_speech_mode, 
            cmd_args.use_lattice_model
        );
    }
}

If ViSQOL is providing unexpected or poor Mean Opinion Score (MOS) predictions, consider the following common causes:

• Low Bitrate/Bandwidth: In audio mode, ViSQOL was trained on full-band audio (up to 24 kHz) with bitrates as low as 24 kbps. If your degraded audio is significantly lower in frequency range or bitrate, performance may degrade.
• Insufficient Audio Activity: ViSQOL requires significant signal activity. It is recommended to use 3 to 10 seconds of audio (ideally 5 seconds) that contains meaningful activity in the reference signal. Too much silence can skew results.
• Use Case Mismatch: ViSQOL is optimized as a proxy for evaluating codecs and VoIP network degradations (similar to ITU-T P.800). While it may work for denoising or generative models, it is not specifically tuned for those tasks.

Solution: If you have subjective scores for your specific use case, consider training a custom SVR model as described in the Support Vector Regression Model Training guide.

The C++ API allows you to perform signal comparisons using the Visqol::VisqolApi class.

Workflow

1. Configure: Create a Visqol::VisqolConfig object. You must set the sample rate (e.g., 48000 Hz) via mutable_audio()->set_sample_rate(). Both signals must share the same sample rate.
2. Initialize: Instantiate Visqol::VisqolApi and call .Create(config). Check the returned absl::Status to ensure success.
3. Measure: Call .Measure(reference_signal, degraded_signal) to get an absl::StatusOr<Visqol::SimilarityResultMsg>.
4. Extract Results: Access metrics from the SimilarityResultMsg object.

Key Configuration Options

• mutable_options()->set_allow_unsupported_sample_rates(bool): Set to true if using non-48k input (not recommended for audio mode).
• mutable_options()->set_model_path(string): Path to the SVR model file.
• mutable_options()->set_use_speech_scoring(bool): Enables speech mode comparison.
• mutable_options()->set_use_unscaled_speech_mos_mapping(bool): If true, uses unscaled speech mode (NSIM 1.0 maps to ~4.x MOS instead of 5.0).

Result Metrics

• moslqo(): Mean Opinion Score - Listening Quality Objective.
• vnsim(): Mean of the frequency band similarity values.
• fvnsim(): Repeated field of similarity results for each frequency band.
• cfb(): Center frequency bands corresponding to fvnsim().
• patch_sims(): A list of PatchSimilarityMsg providing temporal and frequency-specific details for segments of the signal.

int main(int argc, char **argv) {
  Visqol::VisqolConfig config;
  config.mutable_audio()->set_sample_rate(48000);
  config.mutable_options()->set_allow_unsupported_sample_rates(false);
  config.mutable_options()->set_model_path("visqol/model/libsvm_nu_svr_model.txt");
  config.mutable_options()->set_use_speech_scoring(false);
  config.mutable_options()->set_use_unscaled_speech_mos_mapping(false);

  Visqol::VisqolApi visqol;
  absl::Status status = visqol.Create(config);
  if (!status.ok()) {
    std::cout<<status.ToString()<<std::endl;
    return -1;
  }

  absl::StatusOr<Visqol::SimilarityResultMsg> comparison_status_or = visqol.Measure(reference_signal, degraded_signal);
  if (!comparison_status_or.ok()) {
    std::cout<<comparison_status_or.status().ToString()<<std::endl;
    return -1;
  }

  Visqol::SimilarityResultMsg similarity_result = comparison_status_or.value();
  double moslqo = similarity_result.moslqo();
  // ... extract other fields
  return 0;
}

ViSQOL can be used in Python via the visqol_lib_py module.

Installation

Install ViSQOL from the root directory using pip:

pip install .

Usage

1. Create a visqol_config_pb2.VisqolConfig object.
2. Configure the mode (audio or speech):
   • Audio Mode: Set sample_rate to 48000 and use_speech_scoring to False. Use the libsvm_nu_svr_model.txt model.
   • Speech Mode: Set sample_rate to 16000 and use_speech_scoring to True. Use the appropriate .tflite model.
3. Set the svr_model_path in config.options using the absolute path to the model file.
4. Initialize visqol_lib_py.VisqolApi(), call .Create(config), and then .Measure(reference, degraded).
5. The result is a protobuf object containing the moslqo score.

import os
from visqol import visqol_lib_py
from visqol.pb2 import visqol_config_pb2

config = visqol_config_pb2.VisqolConfig()
mode = "audio"
if mode == "audio":
    config.audio.sample_rate = 48000
    config.options.use_speech_scoring = False
    svr_model_path = "libsvm_nu_svr_model.txt"
elif mode == "speech":
    config.audio.sample_rate = 16000
    config.options.use_speech_scoring = True
    svr_model_path = "...tflite"

config.options.svr_model_path = os.path.join(
    os.path.dirname(visqol_lib_py.__file__), "model", svr_model_path)

api = visqol_lib_py.VisqolApi()
api.Create(config)
similarity_result = api.Measure(reference, degraded)
print(similarity_result.moslqo)