DiariZen

Note that this specific repository contains modifications to the original pyannote.audio code for compatibility purposes. If you intend to use the new/original Pyannote code, ensure you are aware of which files have been modified. The modified files in this repository are:

• pyannote/audio/core/model.py
• pyannote/audio/core/inference.py
• pyannote/audio/pipelines/utils/getter.py
• pyannote/audio/pipelines/utils/diarization.py
• pyannote/audio/pipelines/speaker_diarization.py

pyannote.audio does not support streaming speaker diarization out of the box.

If you require streaming capabilities, the following alternatives are recommended:

• diart: A project providing streaming speaker diarization based on pyannote.audio (GitHub).
• Streaming VAD: For streaming Voice Activity Detection, refer to the official blog post on streaming VAD based on pyannote.audio.

To improve the performance of diarization models for your specific use case, follow this workflow:

1. Annotate: Manually annotate dozens of conversations with high precision.
2. Split: Divide your annotated data into training (80%), development (10%), and test (10%) subsets.
3. Format: Set up the data using pyannote.database.
4. Adapt: Follow the recipe for adapting a pretrained pipeline found in tutorials/adapting_pretrained_pipeline.ipynb.

To perform grid searches or launch multiple jobs on a Slurm cluster, use the hydra-submitit-launcher.

1. Install the launcher:

pip install hydra-submitit-launcher --upgrade

2. Launch a multi-run job using the --multirun flag and specifying the submitit_slurm launcher. You can sweep over parameters by providing comma-separated values:

pyannote-audio-train \
    --multirun hydra/launcher=submitit_slurm \
    model=PyanNet +model.lstm.num_layers=2,3,4 +model.lstm.bidirectional=true,false \
    task=VoiceActivityDetection \
    registry="AMI-diarization-setup/pyannote/database.yml" \
    protocol=AMI.SpeakerDiarization.only_words

Warning: There are known compatibility issues between pytorch-lightning, hydra-submitit, and multi-GPU setups.

To install DiariZen, follow these steps to set up a virtual environment, install PyTorch with CUDA 12.1 support, and install the required dependencies including pyannote-audio and dscore submodules.

# create virtual python environment
conda create --name diarizen python=3.10
conda activate diarizen

# install pytorch (CUDA 12.1 build)
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 \
    --index-url https://download.pytorch.org/whl/cu121

# install diarizen 
pip install -r requirements.txt && pip install -e .

# install pyannote-audio
cd pyannote-audio && pip install -e .[dev,testing] -c ../constraints.txt && cd ..

# install dscore
git submodule init
git submodule update

To use pyannote.audio for speaker diarization, follow these steps:

1. Install the package via pip:

   pip install pyannote.audio

2. Access the required models on Hugging Face by accepting the user conditions for:
   • pyannote/segmentation-3.0
   • pyannote/speaker-diarization-3.1
3. Create an access token at hf.co/settings/tokens to authenticate your requests.

Use the DiariZenPipeline class to perform speaker diarization on audio files. You can load pre-trained models from Hugging Face and iterate through the resulting speaker turns.

from diarizen.pipelines.inference import DiariZenPipeline

# load pre-trained model
diar_pipeline = DiariZenPipeline.from_pretrained("BUT-FIT/diarizen-wavlm-large-s80-md")

# apply diarization pipeline
diar_results = diar_pipeline('./example/EN2002a_30s.wav')

# print results
for turn, _, speaker in diar_results.itertracks(yield_label=True):
    print(f"start={turn.start:.1f}s stop={turn.end:.1f}s speaker_{speaker}")

To use the command line tools for training and evaluation, you must install pyannote-audio with the [cli] extra dependencies.

pip install pyannote-audio[cli]

While pyannote.audio uses a gating process (requiring authentication via Hugging Face) to track usage, this does not prevent offline usage in production environments (e.g., inside a docker run command). Once the models or pipelines are authenticated and downloaded, they can be used without repeated authentication. Refer to the following tutorials for specific implementation patterns:

• For models: tutorials/applying_a_model.ipynb
• For pipelines: tutorials/applying_a_pipeline.ipynb

pyannote-audio-train is powered by Hydra. You can override any configuration parameter using dot notation in the command line.

Inspect Configuration

To see the actual configuration being used for a training job, add the --cfg job flag:

pyannote-audio-train --cfg job model=PyanNet task=VoiceActivityDetection ...

Override Parameters

To change a parameter like task.duration, append it to your command:

pyannote-audio-train \
    model=PyanNet \
    task=VoiceActivityDetection task.duration=2.0 \
    registry="AMI-diarization-setup/pyannote/database.yml" \
    protocol=AMI.SpeakerDiarization.only_words

pyannote-audio-train --cfg job \
    model=PyanNet \
    task=VoiceActivityDetection \
    registry="AMI-diarization-setup/pyannote/database.yml" \
    protocol=AMI.SpeakerDiarization.only_words