DOCSEARCHbyAlgolia
Submit Docs

BeatNet Documentation

repository·main·Indexed 19 days ago

https://github.com/mjhydri/beatnet

An AI-based Python library for joint music beat, downbeat, tempo, and meter tracking. BeatNet supports real-time, streaming, online, and offline processing modes using Particle Filtering (PF) and Dynamic Bayesian Network (DBN) inference models. It includes a CRNN model training pipeline, a specialized PyTorch Dataset for audio features, and a particle filter cascade for inferring musical events from activations.

Tokens
5.5K
Snippets
12
Records
21
Agent score
66%

What's inside BeatNet

  1. Train the BeatNet CRNN model

    main

    The training pipeline consists of three steps: data preparation, training, and evaluation.

    1. Prepare Data

    Organize your dataset with .beats annotations (format: <time_in_seconds> <beat_number>, where beat_number == 1 is a downbeat). Use the prepare_data module to extract features.

    2. Train

    Run the training script using a configuration file. You can override parameters like learning_rate, batch_size, and device via CLI.

    3. Use Trained Weights

    Exported weights (best_model_weights.pt) can be loaded directly into the BeatNet inference class using model.load_state_dict().

    # Step 1: Prepare Data
python -m BeatNet.prepare_data --config src/BeatNet/configs/default.yaml \
    --raw_dir /path/to/raw_datasets \
    --dataset BALLROOM GTZAN BEATLES CMR ROCK_CORPUS

# Step 2: Train
python -m BeatNet.train --config src/BeatNet/configs/default.yaml \
    learning_rate=0.001 batch_size=128 device=cuda

# Step 3: Load weights in Python
import torch
from BeatNet.BeatNet import BeatNet

estimator = BeatNet(1, mode='online', inference_model='PF', plot=[])
estimator.model.load_state_dict(
    torch.load('output/best_model_weights.pt', map_location='cpu'), strict=False
)
output = estimator.process("audio_file.wav")

  2. Install BeatNet

    main

    You can install BeatNet via PyPI or directly from the Git repository.

    Prerequisites: Before installing, ensure you have librosa and madmom installed. For audio streaming, pyaudio is required.

    On Mac OS and Linux, you can install pyaudio via pip. On Windows, you may need to download a specific wheel file from here and install it locally using pip.

    # Approach #1: From PyPI
pip install BeatNet

# Approach #2: From Git
pip install git+https://github.com/mjhydri/BeatNet

  3. Format of activations input for particle_filter_cascade

    main

    The process method of particle_filter_cascade expects a numpy array named activations with the following structure:

    • Shape: (num_frames, 2)
    • Column 0: Probabilities/activations corresponding to beats.
    • Column 1: Probabilities/activations corresponding to downbeats.

    The method returns a numpy array of detected events where each row is [timestamp_in_seconds, event_type]:

    • event_type == 1: Downbeat detected.
    • event_type == 2: Beat detected.

  4. Configure BeatNet training hyperparameters

    main

    Training parameters can be set in src/BeatNet/configs/default.yaml or passed as command-line overrides to BeatNet.train.

    | Parameter | Default | Description |
|-----------|---------|-------------|
| `batch_size` | 200 | Training batch size |
| `learning_rate` | 5e-4 | Adam optimizer learning rate |
| `seq_len` | 400 | Training sequence length in frames (8s @ 50fps) |
| `max_epochs` | 10000 | Maximum training epochs |
| `patience` | 20 | Early stopping patience (epochs) |
| `class_weights` | [50, 400, 5] | Cross-entropy weights for [beat, downbeat, non-beat] |
| `checkpoint_every` | 10 | Validate and save every N epochs |
| `val_inference` | DBN | Inference method for validation (DBN or PF) |
| `device` | cpu | Device (cpu, cuda, cuda:0, mps) |

  5. Understand BeatNet working modes

    main

    BeatNet supports four distinct modes of operation depending on your latency and accuracy requirements:

    ModeDescriptionInference Requirement
    'stream'Captures live audio from the system microphone. Best for live performance.Must use 'PF'
    'realtime'Reads an audio file chunk by chunk.Must use 'PF'
    'online'Reads the whole audio and processes it at once.'PF' (causal) or 'DBN' (non-causal)
    'offline'Reads the whole audio and uses a Dynamic Bayesian Network. Faster than standard madmom tracking.Must use 'DBN'

  6. Prepare datasets for BeatNet training

    main

    Use the BeatNet.prepare_data module to transform raw audio and annotation files into processed pickle files (.pkl) ready for model training. The script extracts LOG_SPECT features and builds a one-hot ground truth matrix for beats, downbeats, and non-beat frames.

    Required Raw Directory Structure

    Your raw data must follow this hierarchy:

    {raw_dir}/{dataset_lower}/
    audio/{split_or_genre}/{track}.wav
    annotations/{track}.beats

    Note: {dataset_lower} is the lowercase version of the --dataset name provided.

    python -m BeatNet.prepare_data --config configs/default.yaml --raw_dir /path/to/raw/datasets --dataset BALLROOM

  7. Resume training from a checkpoint

    main

    When resuming training, the script loads the model state, optimizer state, and the training progress (current epoch, best validation F-measure, and early stopping patience counter) from the provided .pt file. This ensures that training can continue seamlessly from where it left off.

    Use the --resume flag to specify the path to the checkpoint file.

    python -m BeatNet.train --config configs/default.yaml --resume output/checkpoint_epoch_100.pt

  8. Train the BeatNet model via CLI

    main

    You can initiate the training pipeline using the BeatNet.train module. The script requires a YAML configuration file and supports overriding configuration values directly from the command line using key=value pairs. You can also resume training from a previously saved checkpoint.

    Basic Usage

    To train using a default configuration:

    python -m BeatNet.train --config src/BeatNet/configs/default.yaml

    Overriding Configuration

    To override specific parameters like learning_rate or batch_size without modifying the YAML file:

    python -m BeatNet.train --config configs/default.yaml learning_rate=0.001 batch_size=128

    Resuming Training

    To resume training from a specific checkpoint file:

    python -m BeatNet.train --config configs/default.yaml --resume output/checkpoint_epoch_100.pt
    python -m BeatNet.train --config src/BeatNet/configs/default.yaml

  9. Fix Madmom compatibility issues

    main

    If using Python >= 3.10 and NumPy >= 1.24, madmom may throw errors.

    Fix 1: Import error In madmom/processors.py, change: from collections import MutableSequence to from collections.abc import MutableSequence

    Fix 2: Numpy alias error Add the following to your sitecustomize.py:

    import numpy as np
if not hasattr(np, 'float'): np.float = np.float64
if not hasattr(np, 'int'): np.int = np.int_

  10. Use BeatNet for music beat and downbeat tracking

    main

    The BeatNet class provides four working modes for different use cases. The system accepts a raw audio waveform object or a directory of audio files. If using a directory, the system automatically resamples audio to 22050 Hz. If providing an audio object, ensure it is already at 22050 Hz.

    Modes:

    • stream: Captures audio directly from the microphone. Use loud input for better performance.
    • realtime: Processes audio files in real-time.
    • online: Processes files faster than real-time using a causal algorithm (identical results to realtime).
    • offline: Infers beats and downbeats non-causally using a Dynamic Bayesian Network (DBN).

    Output: A numpy.array of shape (num_beats, 2) containing columns for beats and downbeats.

    from BeatNet.BeatNet import BeatNet

# Example: Online mode with Particle Filtering (PF)
estimator = BeatNet(1, mode='online', inference_model='PF', plot=['activations'], thread=False)
output = estimator.process("audio_file_directory")

  11. Configure BDObservationModel via observation_lambda

    main

    The BDObservationModel uses the observation_lambda string parameter to determine how the observation model handles the transition between beat/downbeat states and non-beat states. The first character determines the model type:

    • B (Border Model): Followed by an integer (e.g., 'B56'). It classifies a $1/\text{lambda}$ fraction of states as downbeat/beat states and the rest as non-beat states.
    • N (Number Model): Followed by an integer (e.g., 'N3'). It assigns a constant number of beginning states as downbeat/beat states.
    • G (Gaussian Model): Followed by a float (e.g., 'G0.5'). It uses a smooth Gaussian transition (soft border) between states.

  12. Configure BeatNet training via YAML and CLI overrides

    main

    BeatNet uses a YAML-based configuration system. The load_config function merges the contents of a YAML file with command-line overrides.

    Override Syntax

    Overrides are passed as positional arguments in the format key=value. The training script automatically attempts to parse values into their appropriate types:

    • Integers: e.g., batch_size=128
    • Floats: e.g., learning_rate=0.001
    • Booleans: e.g., some_flag=true or some_flag=false (case-insensitive)
    • Strings: Any value that cannot be parsed as the above types remains a string.

    Key Configuration Parameters

    While the exact keys depend on your YAML file, the training script explicitly looks for and uses the following:

    • seed: Random seed for reproducibility.
    • device: Computation device (e.g., 'cpu' or 'cuda').
    • output_dir: Directory where checkpoints and TensorBoard logs are saved.
    • batch_size: Number of samples per training batch.
    • num_workers: Number of subprocesses for data loading.
    • feature_dim: Dimensionality of input features (default: 272).
    • num_cells: Number of cells in the BDA model (default: 150).
    • num_layers: Number of layers in the BDA model (default: 2).
    • learning_rate: Optimizer learning rate.
    • class_weights: List of weights for cross-entropy loss (e.g., [50, 400, 5]).
    • max_epochs: Maximum number of training epochs.
    • patience: Number of epochs to wait for validation improvement before early stopping.
    • checkpoint_every: Frequency of saving checkpoints and running validation.
    • val_inference: The inference method used during validation ('DBN' or 'PF').