3DDFA_V2

The modified BFM2009 face model provided in ../configs/bfm_noneck_v3.pkl is restricted to academic use only.

If you intend to use this model for commercial purposes, you must apply for a commercial license through the official Basel Face Model sources:

1. Basel Face Model (BFM)
2. BFM 2019

Sim3DR is a 3D renderer implemented in C++ and Cython. To build the extension and install it in-place, run the setup.py build command.

python3 setup.py build_ext --inplace

To use 3DDFA_V2, clone the repository and build the required Cython components (NMS, Sim3DR, and the faster mesh render).

Requirements:

• Python 3
• Major dependencies: PyTorch, numpy, opencv-python, and onnxruntime.
• On macOS, if using the --onnx flag for acceleration, install libomp via brew install libomp.

Installation Steps:

1. Clone the repo:

git clone https://github.com/cleardusk/3DDFA_V2.git
cd 3DDFA_V2

2. Build components:

sh ./build.sh

To achieve maximum performance on CPU, use the --onnx flag. You can further control performance by setting the number of OpenMP threads using the OMP_NUM_THREADS environment variable.

Example of setting threads before running:

export OMP_NUM_THREADS=4
python3 demo.py -f examples/inputs/emma.jpg --onnx

The project provides several scripts for different input types. Use the --onnx flag to enable ONNX runtime acceleration, which significantly reduces CPU latency.

• Still Images: Use demo.py. You can specify output modes using the -o flag.
• Videos: Use demo_video.py for standard video processing or demo_video_smooth.py for smoother tracking by looking ahead frames.
• Webcam: Use demo_webcam_smooth.py for real-time webcam tracking.

To use FaceBoxes, you may need to build the CPU version of the Non-Maximum Suppression (NMS) extension. You can do this using the provided build script or the shell script.

# Option 1: Using the build script
cd utils
python3 build.py build_ext --inplace

# Option 2: Using the shell script
sh ./build_cpu_nms.sh

The project uses a simplified version of the Basel Face Model (BFM) named bfm_noneck_v3_slim.pkl. This file is required for 3D dense face alignment tasks. You can download it from the following sources:

• Google Drive: Download Link
• Baidu Drive: Download Link (Password: p803)

3DDFA_V2 supports different model configurations via the -c or --config flag.

• Default: MobileNet_V1 (120x120 input, ~3.27M params).
• Fast/Small: MobileNet x0.5 (120x120 input, ~0.85M params).

Config files are located in the configs/ directory (e.g., configs/mb1_120x120.yml or configs/mb05_120x120.yml).

Before importing the TDDFA modules, ensure that the FaceBoxes and Sim3DR components are successfully built. You can use the provided build script to handle this.

sh build.sh

To run the 3D Dense Face Alignment, you need to load a configuration file and initialize both the FaceBoxes detector and the TDDFA model. It is recommended to use the onnx flag for improved speed. If using ONNX, you may need to set specific environment variables for thread management.

import yaml
import os
from FaceBoxes.FaceBoxes_ONNX import FaceBoxes_ONNX
from TDDFA_ONNX import TDDFA_ONNX
from TDDFA import TDDFA
from FaceBoxes import FaceBoxes

# Load config
cfg = yaml.load(open('configs/mb1_120x120.yml'), Loader=yaml.SafeLoader)

# Recommended: Use ONNX for speed
onnx_flag = True
if onnx_flag:
    os.environ['KMP_DUPLICATE_LIB_OK'] = 'True'
    os.environ['OMP_NUM_THREADS'] = '4'
    face_boxes = FaceBoxes_ONNX()
    tddfa = TDDFA_ONNX(**cfg)
else:
    tddfa = TDDFA(gpu_mode=False, **cfg)
    face_boxes = FaceBoxes()