microWakeWord Documentation

To train a wake word model, micro-wake-word utilizes a combination of synthetic speech, adversarial phrases, and various audio augmentations.

Positive Samples (Wake Word Audio)

• Generated Samples: Uses Piper sample generator for text-to-speech generation and openWakeWord for generating adversarial phrase samples.
• Augmentations: Generated samples are augmented using background audio from FSD50K, FMA, and WHAM!, and reverberated using room impulse responses from the BIRD dataset.

Negative Samples (Ambient Noise)

To reduce false positives, the training process uses ambient noise as negative samples, categorized into:

• Ambient Speech: Derived from the VOICES corpus and Common Voice.
• Ambient Background: Derived from FSD50K, FMA (reverberated), and WHAM!.

The model_train_eval.py script provides a command-line interface to train neural network models (Inception or MixedNet) and evaluate them across different formats (TensorFlow SavedModel, TFLite non-streaming, and TFLite streaming/quantized).

Usage Pattern

1. Train a model: Set --train 1 and specify a model type (inception or mixednet).
2. Evaluate a model: Set --train 0 and enable specific testing flags like --test_tflite_streaming or --test_tflite_nonstreaming_quantized.
3. Resume training: Use --restore_checkpoint 1 to initialize weights and optimizer from an existing checkpoint.

Key CLI Flags

• --training_config: Path to the YAML configuration file.
• --train: Set to 1 to run training, 0 to run only evaluation.
• --restore_checkpoint: Set to 1 to resume interrupted training (requires adjusting learning rate and steps).
• --use_weights: Which weight set to load for evaluation (best_weights or last_weights).
• --verbosity: Log level (INFO, DEBUG, ERROR, FATAL, or WARN).

Model Subcommands

You must provide a model name as a subcommand:

• inception: Uses Inception architecture (parameters defined via inception.model_parameters).
• mixednet: Uses MixedNet architecture (parameters defined via mixednet.model_parameters).

# Example: Train an inception model
python microwakeword/model_train_eval.py --training_config my_config.yaml --train 1 inception

# Example: Evaluate a quantized streaming TFLite model
python microwakeword/model_train_eval.py --training_config my_config.yaml --train 0 --test_tflite_streaming_quantized 1 mixednet

To set up the microWakeWord training environment, install the package and its specific dependencies. Note that certain packages are installed from forks to ensure compatibility with macOS and Jupyter environments.

Important: Restart your Python session after installation is complete.

# For macOS users, install the specific fork for pymicro-features
if platform.system() == "Darwin":
    pip install 'git+https://github.com/puddly/pymicro-features@puddly/minimum-cpp-version'

# Install audio-metadata from a fork to prevent breaking Jupyter
pip install 'git+https://github.com/whatsnowplaying/audio-metadata@d4ebb238e6a401bb1a5aaaac60c9e2b3cb30929f'

# Clone and install microWakeWord in editable mode
git clone https://github.com/kahrendt/microWakeWord
pip install -e ./microWakeWord

import platform

if platform.system() == "Darwin":
    !pip install 'git+https://github.com/puddly/pymicro-features@puddly/minimum-cpp-version'

!pip install 'git+https://github.com/whatsnowplaying/audio-metadata@d4ebb238e6a401bb1a5aaaac60c9e2b3cb30929f'

!git clone https://github.com/kahrendt/microWakeWord
!pip install -e ./microWakeWord

The train function expects a config dictionary. If the following keys are missing, they are assigned default values. Note that many of these keys expect lists to allow for scheduled changes over training steps:

Note: If you provide a list with length $N$, the training loop will iterate through $N$ phases, where each phase lasts for the number of steps specified in training_steps[i].

The training process is controlled by a YAML configuration file. This file defines feature sets, training steps, and optimization hyperparameters.

Feature Set Configuration

Each entry in the features list defines a dataset used during training:

• features_dir: Path to the feature directory.
• sampling_weight: Weight for choosing a sample from this set in a batch.
• penalty_weight: Weight used to penalize incorrect predictions from this set.
• truth: True if the set contains positive (wake word) samples, False if it contains negative (ambient/speech) samples.
• truncation_strategy: How to handle long spectrograms:
  • random: Choose a random portion.
  • truncate_start: Remove the start.
  • truncate_end: Remove the end.
  • split: Split into multiple spectrograms offset by 100ms (recommended for ambient sets).
• type: The data format (e.g., mmap).

Optimization Hyperparameters

• training_steps: Total number of training steps.
• positive_class_weight / negative_class_weight: Lists of weights corresponding to training steps.
• learning_rates: List of learning rates for the Adam optimizer.
• batch_size: Number of samples per batch.
• target_minimization: The target value for the minimization_metric.
• minimization_metric: The metric to minimize (e.g., loss, accuracy, recall, precision, false_positive_rate, false_negative_rate, ambient_false_positives). If set to None, the process only maximizes the maximization_metric.
• maximization_metric: The metric to maximize (e.g., average_viable_recall).