MiVOLO: Multi-input Transformer for Age and Gender Estimation

MiVOLO provides several pretrained models optimized for different tasks and datasets. Models are categorized by their input type (e.g., face_only or face_body) and the attributes they estimate (age, gender, or both).

Available Model Types:

• volo_d1: Typically used for face_only estimation of age and/or gender.
• mivolo_d1: A multi-input model using face_body streams for age and gender estimation.
• mivolov2_d1_384x384: An improved version (MiVOLO v2) using face_body streams, trained on the Lagenda dataset, offering higher accuracy (Age MAE 3.65, Gender Accuracy 97.99% on Lagenda).

MiVOLO utilizes several datasets for training and evaluation. If you use this data, please cite the project's papers.

• Lagenda: Includes images and annotations.
• IMDB-clean: Requires following external instructions for images; annotations are provided.
• UTK: Includes original full images and specific annotations/splits.
• Adience: Requires following external instructions for images; annotations are provided. Uses coarse aligned images from the faces/ directory.
• FairFace: Requires following external instructions for images; annotations are provided. Uses aligned images from the fairface-img-margin125-trainval/ directory.
• AgeDB: Includes images and annotations for ages 1-101.

To run the demo, you must first download the required model weights and place them in the models/ directory:

1. Download the body + face detector model to models/yolov8x_person_face.pt.
2. Download the MiVOLO checkpoint to models/mivolo_imbd.pth.tar.

You can then run the demo on a local image or a YouTube video using the demo.py script.

# Demo for a local image
python3 demo.py \
--input "jennifer_lawrence.jpg" \
--output "output" \
--detector-weights "models/yolov8x_person_face.pt" \
--checkpoint "models/mivolo_imbd.pth.tar" \
--device "cuda:0" \
--with-persons \
--draw

# Demo for a YouTube video
python3 demo.py \
--input "https://www.youtube.com/shorts/pVh32k0hGEI" \
--output "output" \
--detector-weights "models/yolov8x_person_face.pt" \
--checkpoint "models/mivolo_imbd.pth.tar" \
--device "cuda:0" \
--draw \
--with-persons

To reproduce validation metrics, download the prepared annotations for your target dataset (imdb-clean, utk, adience, lagenda, or fairface) and the MiVOLO checkpoint. Use eval_pretrained.py to run the evaluation.

Supported --dataset_name values: utk, imdb, lagenda, fairface, adience.

python3 eval_pretrained.py \
  --dataset_images /path/to/dataset/utk/images \
  --dataset_annotations /path/to/dataset/utk/annotation \
  --dataset_name utk \
  --split valid \
  --batch-size 512 \
  --checkpoint models/mivolo_imbd.pth.tar \
  --half \
  --with-persons \
  --device "cuda:0"

While ONNX export is technically possible, it is not recommended due to poor performance with batch processing. TensorRT and OpenVINO exports are currently not supported because of col2im limitations.

The recommended method for exporting is using TorchScript, which maintains original speed and results in a single file. Use torch.jit.trace to perform the export.

torch.jit.trace(model)

To reproduce the regression benchmark results (Age MAE, Age CS@5, and Gender Accuracy) using the provided checkpoints, use the valid_age_gender.sh script located in the scripts/ directory.

./scripts/valid_age_gender.sh

To install MiVOLO, ensure you have PyTorch 1.13 or newer installed, then install the project dependencies and the package itself using pip.

pip install -r requirements.txt
pip install .