audiomentations

repository·main·Indexed 25 days ago

https://github.com/iver56/audiomentations

A fast Python library for audio data augmentation designed for deep learning training pipelines. It supports mono and multichannel audio (channels-first format) and runs on the CPU. The library provides a wide variety of waveform transforms, such as AddGaussianNoise, PitchShift, and TimeStretch, which can be chained together using a Compose object.

Tokens
21.5K
Snippets
34
Records
106
Agent score
80%

What's inside audiomentations

  1. Compare Audiomentations with alternative audio augmentation libraries

    main

    If Audiomentations does not meet your specific requirements (such as GPU acceleration or specific framework integration), you may consider these alternative Python libraries for audio data augmentation and degradation:

    GPU-Supported Alternatives

    • audiotools: Supports GPU.
    • fast-audiomentations: Supports GPU.
    • kapre: Supports GPU (Keras/Tensorflow).
    • SpecAugment: Supports GPU (Pytorch & Tensorflow).
    • spec_augment: Supports GPU (Pytorch).
    • teal: Supports GPU (Keras/Tensorflow).
    • torch-audiomentations: Supports GPU (Pytorch).
    • torchaudio-augmentations: Supports GPU (Pytorch).
    • torchfx: Supports GPU (Pytorch).

    CPU-Only Alternatives

    • audio-degradation-toolbox
    • audio_degrader
    • auglib
    • AugLy
    • muda
    • nlpaug
    • pedalboard
    • pydiogment
    • python-audio-effects
    • wav2aug
    • WavAugment
  2. Use `AddBackgroundNoise` to mix in background sounds

    main

    The AddBackgroundNoise transform mixes an external sound (e.g., background noise, music) into your input audio. This is useful for simulating real-world environments or performing 'mixup' for training classification models.

    Key Behaviors

    • Sound Length: Background sounds should ideally be at least as long as the input audio. If they are shorter, they will be repeated, which may sound unnatural.
    • Silence Handling: When using the default noise_rms="relative" mode, if the input audio is completely silent, no noise will be added because the noise level is proportional to the input signal.
    • Noise Transformation: You can optionally apply other transforms to the noise itself using the noise_transform parameter before it is mixed into the main signal.
    from audiomentations import AddBackgroundNoise
    
    transform = AddBackgroundNoise(sounds_path="/path/to/noise")
    augmented_sound = transform(my_waveform_ndarray, sample_rate=16000)
  3. How Compose, OneOf, and SomeOf work together

    main

    Audiomentations uses composition classes to manage how transforms are applied:

    • Compose: Applies a sequence of transforms in order. It can optionally shuffle the sequence for every call.
    • OneOf: Randomly selects exactly one transform from a provided list to apply. You can provide a weights list of floats to bias the selection; otherwise, it chooses uniformly at random.
    • SomeOf: Randomly selects several transforms from a provided list to apply.
    from audiomentations import OneOf, PitchShift
    
    pitch_shift = OneOf(
        transforms=[
            PitchShift(method="librosa_phase_vocoder"),
            PitchShift(method="signalsmith_stretch"),
        ],
        p=1.0,
        weights=[0.1, 0.9],
    )
  4. How RepeatPart modes affect audio length

    main

    RepeatPart provides two modes that change how the output waveform relates to the input waveform:

    • mode="insert": The repeats are added to the signal. The output array is longer than the input. The original audio that followed the selected part is shifted later in time.
    • mode="replace": The repeats overwrite the selected part of the original signal. The output array is the same length as the input. Any audio at the end of the file that was not part of the selection remains unchanged and unshifted.
  5. Configure Mp3Compression backends

    main

    The backend parameter determines how the compression is performed.

    • "fast-mp3-augment" (Recommended): In-memory computation with parallel threads. Uses LAME encoder and minimp3 decoder. It is the fastest option and avoids temporary files.
    • "pydub" (Deprecated): Uses pydub + ffmpeg. It is slower because it writes temporary files to disk. Note: This backend is deprecated as of v0.43.0 due to dependencies on the deprecated audioop module.
    • "lameenc" (Deprecated): Slow; writes a temporary file to disk. It introduces encoder/decoder delay and does not support preserve_delay=False. Note: This backend is deprecated as of v0.42.0 because "fast-mp3-augment" is up to 4x faster.
  6. Deciding between CPU and GPU for audio data augmentation

    main

    When designing an audio machine learning training pipeline, you must choose whether to run augmentation transforms on the CPU (using audiomentations) or the GPU (using libraries like torch-audiomentations). The decision depends on where your training bottleneck lies.

    Use CPU-only audiomentations if:

    • You want to maximize VRAM for models: CPU augmentation does not consume VRAM, allowing you to use larger batch sizes.
    • Your GPU is already heavily utilized: If your model's GPU utilization is high, running augmentations on multiple CPU threads via a data loader is often sufficient to keep the GPU busy without creating a bottleneck.
    • You need a wider variety of transforms: audiomentations offers a more extensive selection of transforms, including those that only have CPU implementations (e.g., Mp3Compression).
    • You need library independence: audiomentations is not tied to specific tensor libraries like TensorFlow or PyTorch.
    • You want simplicity: It is straightforward to install and prototype with.

    Use GPU-accelerated transforms if:

    • Your model is small: If your model does not fully utilize the GPU's processing power or VRAM, running transforms on the GPU can be more efficient.
    • CPU data loading is the bottleneck: If your CPU-based data loader cannot keep up with the GPU, moving transforms to the GPU can speed up training.
    • You are using specific heavy transforms: Certain operations like convolution (used for room reverb or filters) can be significantly faster on a GPU.
  7. Use BandStopFilter for audio augmentation

    main

    The BandStopFilter (also known as a notch filter or band reject filter) applies band-stop filtering to input audio. This is useful for preventing models from overfitting to specific frequency relationships and making them robust to frequency losses in diverse audio environments.

    Key characteristics:

    • The center frequency is picked in mel space, aligning it with human hearing.
    • Filter steepness (roll-off) is parameterized in dB/octave.
    • It can be configured for zero-phase filtering to avoid phase distortion, which is particularly useful for audio with many transients (e.g., drum tracks).
  8. Configure Limiter threshold modes

    main

    The Limiter transform supports two modes for determining the threshold level via the threshold_mode parameter:

    • "relative_to_signal_peak": The threshold is calculated relative to the peak of the input signal. (Default)
    • "absolute": The threshold is relative to 0 dBFS and does not depend on the signal's peak.
    from audiomentations import Limiter
    
    # Example: Absolute threshold
    transform = Limiter(
        min_threshold_db=-16.0,
        max_threshold_db=-6.0,
        threshold_mode="absolute",
        p=1.0,
    )
    
    augmented_sound = transform(my_waveform_ndarray, sample_rate=16000)
  9. Use SevenBandParametricEQ for frequency spectrum augmentation

    main

    The SevenBandParametricEQ transform adjusts the volume of seven different frequency bands. It uses a combination of one low shelf filter, five peaking filters, and one high shelf filter. All filters are applied with randomized gains, Q values, and center frequencies within predefined ranges.

    This transform is useful for making machine learning models more robust to variations in frequency spectra caused by microphone types, room acoustics, or sound source quality, as it changes the timbre while preserving the overall 'class' of the sound.

  10. Use GainTransition for fade in and fade out

    main

    The GainTransition transform gradually changes the volume up or down over a random time span, simulating fade-in and fade-out effects. The transition follows a logarithmic scale to match human hearing.

    How it works:

    1. The transform selects two gain levels (a starting gain and an ending gain).
    2. It selects a random duration for the transition between these two gains.
    3. The gain is held constant at the first level until the transition starts, then moves to the second level, and is held constant at that level until the audio ends.

    Note: The transition can start before the audio begins or end after the audio ends, meaning the output audio may capture only a portion of the transition.

  11. Configure zero-phase filtering in BandStopFilter

    main

    Setting zero_phase=True ensures that the filtering does not affect the phase of the input signal. This is recommended when augmenting audio files with significant transients, such as drum tracks.

    Note the following trade-offs when using zero_phase=True:

    • Attenuation: It results in a 3 dB drop at the cutoff frequencies (compared to 6 dB in non-zero phase mode).
    • Performance: It is approximately twice as slow as non-zero phase filtering.
    • Roll-off constraints: The min_rolloff and max_rolloff values must be multiples of 12 (instead of 6) when zero_phase is True.
  12. Multichannel audio support

    main

    As of v0.22.0, most transforms support multichannel audio.

    • Mono: 1-dimensional numpy arrays.
    • Stereo/Multichannel: 2D arrays with shape (num_channels, num_samples).

    Exceptions: AddBackgroundNoise and AddShortNoises do not currently support multichannel audio.