Resemblyzer Documentation

Resemblyzer is a deep learning-based tool for voice representation. Key features include:

• Voice Similarity: Provides a metric to compare how similar different voices sound.
• High-level Feature Extraction: Generates 256-dimensional embeddings that can be used for voice cloning, component analysis (prosody, gender, etc.), or as feature vectors for ML.
• PyTorch Integration: The voice encoder is written in PyTorch, allowing it to be used as a perceptual loss function in deep learning training via backpropagation.
• Performance: Fast execution (approx. 1000x real-time on a GTX 1080) and supports both CPU and GPU.
• Robustness: Robust to noise and works best on English, though it supports other languages to a degree.

To install the Resemblyzer package, use pip. Note that Python 3.5 or higher is required.

pip install resemblyzer

You can derive a high-level representation of a voice (a 256-value embedding) by using the VoiceEncoder class. First, preprocess the audio file using preprocess_wav, then pass the processed waveform to encoder.embed_utterance().

Applications include:

• Speaker verification: Compare a voice profile to new audio.
• Speaker diarization: Identify who is speaking in multi-speaker segments.
• Fake speech detection: Compare similarity of utterances against ground truth.
• Feature extraction: Use embeddings for machine learning or component analysis (accents, gender, etc.).

from resemblyzer import VoiceEncoder, preprocess_wav
from pathlib import Path
import numpy as np

fpath = Path("path_to_an_audio_file")
wav = preprocess_wav(fpath)

encoder = VoiceEncoder()
embed = encoder.embed_utterance(wav)
np.set_printoptions(precision=3, suppress=True)
print(embed)