VoxCeleb Trainer

To speed up training, use the following flags:

Mixed Precision

Use the --mixedprec flag. This is recommended for Tesla V100, GeForce RTX 20 series, or later GPUs.

Distributed Training

Use the --distributed flag.

• GPU Selection: Set your target GPUs using the environment variable export CUDA_VISIBLE_DEVICES=0,1,2,3 before running the training command.
• Port Configuration: If running multiple distributed training sessions simultaneously, you must specify a different --port argument to avoid conflicts.

export CUDA_VISIBLE_DEVICES=0,1,2,3
python ./trainSpeakerNet.py --distributed --mixedprec ...

To install the necessary Python dependencies, use pip with the provided requirements.txt. The project requires Python version 3.8 or higher. Additionally, ensure that wget and ffmpeg are installed on your system.

pip install -r requirements.txt

Use the dataprep.py script to download, concatenate, extract, and convert the VoxCeleb dataset. Follow this sequence of commands:

1. Download and prepare: python ./dataprep.py --save_path data --download --key ACCESS_KEY
2. Concatenate: python ./dataprep.py --save_path data --concatenate
3. Extract: python ./dataprep.py --save_path data --extract
4. Convert: python ./dataprep.py --save_path data --convert

If you intend to use data augmentation, run the following command as well:

python ./dataprep.py --save_path data --augment

You can evaluate pretrained models using the --eval flag. Below are examples to verify specific model performance.

ResNetSE34 (Baseline Lite AP)

Expected EER: 2.1792

python ./trainSpeakerNet.py --eval --model ResNetSE34L --log_input --trainfunc angleproto --save_path exps/test --eval_frames 400 --initial_model baseline_lite_ap.model

ResNetSE34V2 (Baseline V2 SMProto)

Expected EER: 1.0180

python ./trainSpeakerNet.py --eval --model ResNetSE34V2 --log_input --encoder_type ASP --n_mels 64 --trainfunc softmaxproto --save_path exps/test --eval_frames 400  --initial_model baseline_v2_smproto.model

RawNet3

Expected EER: 0.8932

python ./trainSpeakerNet.py --eval --config ./configs/RawNet3_AAM.yaml --initial_model models/weights/RawNet3/model.pt

The trainer implements several model architectures. Use the following citations when referencing these specific models:

• VGGVox: Nagrani et al. (2017)
• ResNet: He et al. (2016)

@inproceedings{nagrani2017voxceleb,
  title={VoxCeleb: A Large-Scale Speaker Identification Dataset},
  author={Nagrani, Arsha and Chung, Joon Son and Zisserman, Andrew},
  booktitle={Interspeech},
  pages={2616--2620},
  year={2017}
}

@inproceedings{he2016deep,
  title={Deep residual learning for image recognition},
  author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
  pages={770--778},
  year={2016}
}

The following components are available for use in training configurations:

Implemented Loss Functions

• softmax (Softmax)
• amsoftmax (AM-Softmax)
• aamsoftmax (AAM-Softmax)
• ge2e (GE2E)
• proto (Prototypical)
• triplet (Triplet)
• angleproto (Angular Prototypical)

Implemented Models and Encoders

• ResNetSE34L: Supports SAP, ASP encoders.
• ResNetSE34V2: Supports SAP, ASP encoders.
• VGGVox40: Supports SAP, TAP, MAX encoders.

If you use this code in your research, please cite the following paper:

@inproceedings{chung2020in,
  title={In defence of metric learning for speaker recognition},
  author={Chung, Joon Son and Huh, Jaesung and Mun, Seongkyu and Lee, Minjae and Heo, Hee Soo and Choe, Soyeon and Ham, Chiheon and Jung, Sunghwan and Lee, Bong-Jin and Han, Icksang},
  booktitle={Interspeech},
  year={2020}
}

The trainer uses the following databases/methods for data augmentation:

• MUSAN database: Snyder et al. (2015)
• Room Impulse Response database: Ko et al. (2017)

@article{snyder2015musan,
  title={Musan: A music, speech, and noise corpus},
  author={Snyder, David and Chen, Guoguo and Povey, Daniel},
  journal={arXiv preprint arXiv:1510.08484},
  year={2015}
}

@inproceedings{ko2017study,
  title={A study on data augmentation of reverberant speech for robust speech recognition},
  author={Ko, Tom and Peddinti, Vijayaditya and Povey, Daniel and Seltzer, Michael L and Khudanpur, Sanjeev},
  booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing},
  pages={5220--5224},
  year={2017}
}

The trainer includes aggregation methods for pooling features. Use the following citations:

• SAP (Statistics Pooling): Bhattacharya et al. (2017)
• ASP (Attentive Statistics Pooling): Okabe et al. (2018)

@inproceedings{bhattacharya2017deep,
  title={Deep Speaker Embeddings for Short-Duration Speaker Verification},
  author={Bhattacharya, Gautam and Alam, Md Jahangir and Kenny, Patrick},
  booktitle={Interspeech},
  pages={1517--1521},
  year={2017}
}

@inproceedings{okabe2018attentive,
  title={Attentive Statistics Pooling for Deep Speaker Embedding},
  author={Okabe, Koji and Koshinaka, Takafumi and Shinoda, Koichi},
  booktitle={Interspeech},
  pages={2252--2256},
  year={2018}
}