The library provides several pre-trained models based on two different training datasets: MTT (MagnaTagATune) and MSD (Million Song Dataset).

musicnn models

These are musically motivated CNNs with a dense mid-end and temporal-pooling back-end.

• 'MTT_musicnn': Trained on the MTT dataset.
• 'MSD_musicnn': Trained on the MSD dataset.
• 'MSD_musicnn_big': A larger version of the MSD model with more capacity (x512 filters in the mid-end and 500 units in the back-end). Requires installation from source (python setup.py install).

vgg models

These are baseline models adapted from computer vision CNNs (VGG architecture) trained on audio-spectrograms.

• 'MTT_vgg': Trained on the MTT dataset.
• 'MSD_vgg': Trained on the MSD dataset.
• Note: vgg models do not support custom input_length; they must be set to 3.

The musicnn library uses librosa for audio reading. librosa primarily relies on soundfile, falling back to audioread for codecs not supported by soundfile (such as MP3 and certain WAV variants).

To ensure compatibility, verify that your audio files use codecs supported by libsndfile.

When performing transfer learning with musicnn, the following layers are recommended based on preliminary experiments and visualizations:

• taggram: Provides high-level music information.
• max_pool: Provides a relatively sparse acoustic representation of the music audio.

You can install musicnn via pip for standard usage. If you require larger models or want to access the full documentation and Jupyter notebooks, install directly from the source repository.

pip install musicnn

To install from source for larger models and documentation:

git clone https://github.com/jordipons/musicnn.git
python setup.py install

A Taggram represents the temporal evolution of tag likelihoods. To derive a single song-level likelihood for each tag, you can average the Taggram values across the time dimension using numpy.mean.

import numpy as np

# Assuming taggram is the output from musicnn.extractor.extractor
# averaging the Taggram through time
tags_likelihood_mean = np.mean(taggram, axis=0)

When extract_features=True is used, the features dictionary contains tensors representing the output of different layers.

Tensor Dimensions

Most features are 3D tensors with dimensions: (time, frequency, #filters).

Layer Specifics

• pool1 through pool4: These are 3D tensors. For example, features['pool1'].shape might return (time, frequency, filters).
• pool5: This layer is unique because the max-pooling operators have summarized the frequency content into a single value. Instead of a 3D tensor, musicnn returns a 2D representation with dimensions: (time, #filters).

Model Architecture Notes

• All CNN layers use same padding.
• All max-pooling layers use a stride of 2x2, except for the last max-pooling layer which uses a stride of 4x4.

If your model is running slowly even when using a GPU, you can increase the processing speed by setting a larger batch size. By default, the batch size is set to 1 in ./musicnn/configuration.py to ensure computational safety, but it can be increased for better performance.

# In ./musicnn/configuration.py
BATCH_SIZE = 1  # Increase this value to improve speed

The Taggram represents the temporal evolution of musical tags. You can compute it using the extractor function from musicnn.extractor. This returns both the taggram data and the list of tags used.

from musicnn.extractor import extractor

taggram, tags = extractor('./audio/example.mp3', model='MTT_musicnn')

Use the top_tags function from musicnn.tagger to estimate the most prominent musical tags for an audio file. You can specify the model and the number of tags (topN) to return.

from musicnn.tagger import top_tags

# Predict top 10 tags using the 'MTT_musicnn' model
tags = top_tags('./audio/example.mp3', model='MTT_musicnn', topN=10)
print(tags)