FaceRecognitionDotNet

The Head Pose Training program estimates face pose (Roll, Pitch, Yaw) using 68 face landmarks.

The Process:

1. Landmark Centering: Point 34 is used as the center of the face.
2. Vector Calculation: Calculate the distance from point 34 to all other 67 points (1-68, excluding 34).
3. Normalization: Apply z-score normalization to the vectors (except for roll).
4. Pattern Extraction:
   • Roll: Uses the arc tangent of X and Y distances.
   • Pitch: Uses the euclidean distance of Y only.
   • Yaw: Uses the euclidean distance of X only.
5. Training: The resulting three 67-dimensional vectors are trained using the kernel recursive least squares algorithm.

When working with raw bitmap data, you must ensure the colorspace matches the expected input of the library.

• FaceRecognitionDotNet.LoadImageFile returns a FaceRecognitionDotNet.Image object that uses the RGB colorspace, following the face_recognition specification and dlib requirements.
• FaceRecognitionDotNet.LoadImage accepts raw bitmap data, but this data may be in BGR colorspace (for example, 24-bit Windows bitmap file data accessed via System.Drawing.Bitmap.LockBits and Scan0).

Warning: Using the wrong colorspace leads to significant errors. Specifically, FaceRecognitionDotNet.FaceRecognition.FaceEncodings produces different results for RGB vs BGR images. Even if the face location is identical, the resulting FaceRecognitionDotNet.FaceEncoding values will have an unacceptable distance between them.

Run the training process using the train command. Note that the dataset may contain invalid classes, which will result in fewer training/test samples than expected.

Available Arguments:

• --dataset: The directory of the dataset to use.
• --epoch: Total number of epochs.
• --lr: Initial learning rate.
• --min-lr: Minimum learning rate.
• --min-batchsize: Minimum batch size.
• --validation-interval: Interval for validation.

To build the Age Training program, navigate to the <AgeTraining_dir> in your command prompt and execute the following commands to remove existing references, add the FaceRecognitionDotNet package, and build in Release mode:

Important Requirements:

• You must copy DlibDotNetNative.dll, DlibDotNetNativeDnn.dll, and DlibDotNetNativeDnnAgeClassification.dll to the output directory: <AgeTraining_dir>\bin\Release\netcoreapp2.0.
• For optimal performance, build the required DlibDotNetNative DLLs with CUDA support.
• If running on Linux or MacOS, you must build DlibDotNet first (refer to the DlibDotNet wiki for platform-specific tutorials).

$ dotnet remove reference ../../src/FaceRecognitionDotNet\FaceRecognitionDotNet.csproj
$ dotnet add package FaceRecognitionDotNet
$ dotnet build -c Release

FaceRecognitionDotNet provides several NuGet packages depending on your target operating system, CPU architecture, and whether you want to use GPU acceleration (CUDA) or Intel MKL.

Important: Choose the package that matches your hardware and CUDA version if using GPU acceleration.

Due to licensing restrictions, FaceRecognitionDotNet does not provide pretrained model files. You must train the datasets yourself to use features like Age, Emotion, Gender, or Head Pose prediction.

Training tools are provided in the repository under the tools/ directory for the following tasks:

• AgeTraining
• EmotionTraining / EmotionTrainingV2
• GenderTraining
• HeadPoseTraining
• HelenTraining (for landmarks)

The Age Training tool requires specific datasets and model files to function.

1. Dataset

Download the Adience dataset from this URL (requires registration). You need to extract and copy the following files to <AgeTraining_dir>:

• aligned.tar.gz
• faces.tar.gz
• fold_0_data.txt through fold_4_data.txt
• fold_frontal_0_data.txt through fold_frontal_4_data.txt

2. Model File

Download and extract the shape predictor file to <AgeTraining_dir>:

• http://dlib.net/files/shape_predictor_5_face_landmarks.dat.bz2

The Gender Training tool provides Python scripts to visualize the dataset distribution and training progress.

Dataset Visualization:

• To check age distribution by gender: python tools\age-by-gender-hist.py <DatasetDirectory>
• To check gender distribution: python tools\gender-hist.py <DatasetDirectory>

Training Log Visualization:

• To visualize the training log as a graph: python tools\visualize-log.py <LogFile>

# Check dataset status
python tools\age-by-gender-hist.py UTKFaceDataset
python tools\gender-hist.py UTKFaceDataset

# Check training log
python tools\visualize-log.py utkface-gender-network_600_0.001_1E-05_400.log

Before training, you must generate training and testing image lists from your raw data directory (e.g., a 300W-LP directory). Use the tools/GenerateTrainTestList.ps1 PowerShell script to create these lists in your target <dataset_dir>.

Use the -trainingRatio flag to determine the split (e.g., 0.9 for 90% training and 10% testing).

To build the OpenCVSharp sample, you must first ensure you have the necessary model files prepared. Then, navigate to the <OpenCVSharpSample_dir> in your command prompt and execute the following commands to clean up existing references, add the FaceRecognitionDotNet package, and build the project in Release mode.

$ dotnet remove reference ../../src/FaceRecognitionDotNet\FaceRecognitionDotNet.csproj
$ dotnet add package FaceRecognitionDotNet
$ dotnet build -c Release