DOCSEARCHbyAlgolia
Submit Docs

EfficientAT

repository·main·Indexed 43 days ago

https://github.com/fschmid56/efficientat

Efficient pre-trained CNN models for audio pattern recognition designed for resource-constrained platforms. It utilizes Knowledge Distillation from Transformers to provide high performance with low computational and parameter complexity, featuring pre-trained MobileNet (MN) and Dynamic MobileNet (DyMN) architectures.

Tokens
3K
Snippets
12
Records
14
Agent score
7%

What's inside EfficientAT

  3. Fine-tune on ESC-50

    main

    Prepare the ESC50 dataset following PaSST instructions. You need an esc50 folder containing a meta folder (with meta.csv) and an audio_32k folder (with .wav files).

    Set the directory path in dataset_dir within datasets/esc50.py. Use ex_esc50.py for fine-tuning. Since ESC-50 is divided into 5 cross-validation folds, use the --fold flag to specify which fold to use for testing.

    python ex_esc50.py --cuda --pretrained --model_name=mn10_as --fold=1

  4. Train models on AudioSet using ex_audioset.py

    main

    Use ex_audioset.py to train various model architectures on AudioSet. Common flags include --cuda for GPU acceleration, --train to initiate training, --pretrained to use pre-trained weights, and --model_name to specify the architecture (e.g., mn10_im for MobileNet or dymn10_im for DyMN).

    Key configuration options:

    • --batch_size: Number of samples per batch.
    • --max_lr: Maximum learning rate.
    • --model_width: Controls model scale (e.g., 0.1 for tiny models).
    • --head_type: Type of classification head (e.g., fully_convolutional).
    • --se_dims: Squeeze-and-Excitation dimensions (e.g., f for frequency dimension).
    • --adamw: Enables AdamW optimizer.
    • --weight_decay: Sets weight decay value.
    • --pretrain_final_temp: Sets temperature for pre-training.
    python ex_audioset.py --cuda --train --pretrained --model_name=mn10_im --batch_size=60 --max_lr=0.0004

  5. Fine-tune for Acoustic Scene Classification (DCASE 2020)

    main

    Fine-tuning for acoustic scene classification requires the TAU Urban Acoustic Scenes 2020 Mobile, Development dataset. The directory must contain an audio/ folder (with .wav files), meta.csv, and an evaluation_setup/ folder.

    Set the directory path in dataset_dir within datasets/dcase20.py. Use ex_dcase20.py for training. It is highly recommended to use the --cache_path flag to store resampled waveforms and prevent performance bottlenecks.

    python ex_dcase20.py --cuda --pretrained --model_name=mn10_as --cache_path=cache

  6. Fine-tune on OpenMIC-2018

    main

    Prepare the OpenMIC-2018 dataset following PaSST instructions. You need a directory containing openmic_train.csv_mp3.hdf and openmic_test.csv_mp3.hdf.

    Set the directory path in dataset_dir within datasets/fsd50k.py. Use ex_openmic.py for fine-tuning.

    python ex_openmic.py --cuda --train --pretrained --model_name=mn10_as

  7. Train and Evaluate on AudioSet

    main

    To use AudioSet, you must first prepare the dataset following the PaSST repository instructions. You need three specific .hdf files: balanced_train_segmenets_mp3.hdf, unbalanced_train_segmenets_mp3.hdf, and eval_segmenets_mp3.hdf.

    After downloading, set the path to the folder containing these files in the dataset_dir variable within datasets/audioset.py.

    Training and evaluation are performed using ex_audioset.py. Logging is integrated with Weights & Biases, and the latest models are saved to the wandb directory.

    python ex_audioset.py --cuda --model_name="dymn10_as"

  8. Fine-tune on FSD50K

    main

    To fine-tune on FSD50K, prepare the dataset using the PaSST repository instructions. You need a directory containing FSD50K.train_mp3.hdf, FSD50K.val_mp3.hdf, and FSD50K.eval_mp3.hdf.

    Set the location of this directory in dataset_dir within datasets/fsd50k.py. Use ex_fsd50k.py for fine-tuning.

    python ex_fsd50k.py --cuda --train --pretrained --model_name=mn10_as

  9. Configure the default directory for W&B experiment logging and model storage

    main
    The wandb/ directory is used by default to log experiments and store trained models. Ensure this directory is accessible in your environment to allow the project to persist experiment metadata and model checkpoints.

  10. Run inference on an audio file

    main

    Use the inference.py script to perform inference on an audio file.

    Important: All models are trained with half precision (float16). Running float32 inference on a CPU may result in slight performance degradation.

    To run inference with a single model (e.g., dymn10_as) using CUDA:

    python inference.py --cuda --model_name=dymn10_as --audio_path="resources/metro_station-paris.wav"

  11. Run inference using a model ensemble

    main

    You can perform inference using an ensemble of multiple models by passing them to the --ensemble flag in inference.py.

    python inference.py --ensemble dymn20_as mn40_as_ext mn40_as --cuda --audio_path=resources/metro_station-paris.wav

  12. Load pre-trained Dynamic MobileNet models

    main

    You can load pre-trained Dynamic MobileNet (DyMN) models using the get_model function from models.dymn.model. The naming convention uses the dymn prefix (e.g., dymn10_as is a dynamic MobileNet with width_mult=1.0 pre-trained on AudioSet).

    from models.dymn.model import get_model as get_dymn
model = get_dymn(pretrained_name="dymn10_as")