DOCSEARCHbyAlgolia
Submit Docs

@vladmandic/face-api

repository·master·Indexed 22 days ago

https://github.com/vladmandic/face-api

An AI-powered library for face detection, rotation tracking, recognition, age/gender/emotion prediction, and face description. Built for Browser and NodeJS environments using TensorFlow.js, it supports WebGL, CPU, WASM, and CUDA backends. It provides a 68-point face mesh for facial geometry analysis and includes tools for face alignment via the FaceLandmarks class. This version updates the original face-api.js to support tfjs >= 2.0 and introduces ESM and CommonJS module systems.

Tokens
13.1K
Snippets
47
Records
75
Agent score
77%

What's inside @vladmandic/face-api

  1. Important: Migration to Human library

    master
    While @vladmandic/face-api is maintained, it is completely superseded by the newer library Human. Human covers the same use cases but extends them with newer AI models, additional detection details, and compatibility with the latest web standards. If you are starting a new project, consider using Human instead.

  2. FaceAPI Features Overview

    master

    The face-api library provides several computer vision capabilities for facial analysis, including:

    • Face Recognition: Identifying or verifying specific faces.
    • Face Landmark Detection: Locating specific facial features (eyes, nose, mouth, etc.).
    • Face Expression Recognition: Detecting emotional states.
    • Age Estimation & Gender Recognition: Predicting age and gender from facial images.

  4. Choose a Face Landmark Detection Model

    master

    For detecting 68 facial landmark points, you can choose between two lightweight models:

    • Standard Model (face_landmark_68_model): Balanced accuracy and speed. Size: ~350 KB.
    • Tiny Model (face_landmark_68_tiny_model): Optimized for maximum speed and minimum footprint. Size: ~80 KB.

  5. Use the Face Recognition Model

    master

    The face recognition model uses a ResNet-34 like architecture to generate a face descriptor (a 128-value feature vector) for any given face.

    Key usage details:

    • Generalization: The model is not limited to the faces it was trained on; it can recognize any person.
    • Comparison: To determine if two faces belong to the same person, compare their descriptors using Euclidean distance or a similar classifier.
    • Accuracy: Achieves 99.38% accuracy on the LFW benchmark.
    • Model ID: face_recognition_model.
    • Size: ~6.2 MB.

  6. Choose a Face Detection Model

    master

    The project provides two primary models for face detection, allowing you to trade off accuracy for performance depending on your target environment:

    1. SSD Mobilenet V1 (ssd_mobilenetv1_model):

      • Best for: High accuracy requirements.
      • Characteristics: Computes bounding boxes and probabilities for each face. It prioritizes accuracy over inference speed.
      • Size: ~5.4 MB.

    2. Tiny Face Detector (tiny_face_detector_model):

      • Best for: Mobile devices, web browsers, and resource-constrained clients.
      • Characteristics: Real-time, high performance, and very fast. It is less effective at detecting very small faces but produces bounding boxes that better cover facial feature points, making it ideal for subsequent landmark detection.
      • Size: ~190 KB.

  7. Key differences in @vladmandic/face-api vs original face-api.js

    master

    Compared to the original face-api.js (v0.22.2), this version provides several critical improvements for modern development:

    • TensorFlow/JS Compatibility: Supports tfjs >= 2.0 (currently using 4.16), whereas the original is locked to 1.7.4.
    • Backend Support: Compatible with WebGL, CPU, and WASM for browsers, and tfjs-node / tfjs-node-gpu for NodeJS.
    • Module Systems: Switched from UMD to ESM + CommonJS with IIFE fallback. ESM imports are fully tree-shakable.
    • Bundling Options: Offers versions with tfjs pre-bundled and a -nobundle version for users who want to manage tfjs versions manually.
    • New Features: Includes face angle calculations (roll, yaw, and pitch) and a version class to check FaceAPI and linked TFJS versions.
    • Model Changes: mtcnn and tinyYolov2 models have been removed. Valid models are tinyFaceDetector and mobileNetv1.

  8. Use Face Expression and Age/Gender Recognition Models

    master

    In addition to detection and recognition, the library supports:

    • Face Expression Recognition (face_expression_recognition_model):

      • Lightweight and fast.
      • Note: Accuracy may decrease if the subject is wearing glasses.
      • Size: ~310 KB.

    • Age and Gender Recognition:

      • A multitask network providing age regression and gender classification.
      • Performance: Average Mean Age Error (MAE) of ~4.54 and Gender Accuracy of ~95% across tested databases.
      • Size: ~420 KB.

  9. Display detection results on a canvas

    master

    To visualize results, you must first prepare an overlay canvas that matches the dimensions of your input media.

    1. Match Dimensions: Use faceapi.matchDimensions(canvas, displaySize) where displaySize is { width: input.width, height: input.height }.
    2. Resize Results: Since detections are relative to the original image, use faceapi.resizeResults(detections, displaySize) before drawing.
    3. Draw: Use the faceapi.draw namespace for high-level drawing functions:
      • drawDetections(canvas, resizedDetections)
      • drawFaceLandmarks(canvas, resizedResults)
      • drawFaceExpressions(canvas, resizedResults, [minProbability])

    Custom Drawing:

    • faceapi.draw.DrawBox(box, options): For custom bounding boxes.
    • faceapi.draw.DrawTextField(text, anchor, options): For custom text overlays.
    const displaySize = { width: input.width, height: input.height }
const canvas = document.getElementById('overlay')
faceapi.matchDimensions(canvas, displaySize)

const detections = await faceapi.detectAllFaces(input)
const resizedDetections = faceapi.resizeResults(detections, displaySize)

faceapi.draw.drawDetections(canvas, resizedDetections)

  10. Perform face recognition with FaceMatcher

    master

    Face recognition is achieved by comparing a query face descriptor against a set of reference descriptors using the FaceMatcher class.

    1. Initialize Reference Data: Detect faces in a reference image and extract their descriptors.
    2. Create Matcher: Pass the detection results to new faceapi.FaceMatcher(results) to automatically assign labels.
    3. Match Query: Use faceMatcher.findBestMatch(descriptor) with a descriptor from a new image.
    // 1. Setup reference matcher
const results = await faceapi
  .detectAllFaces(referenceImage)
  .withFaceLandmarks()
  .withFaceDescriptors()

const faceMatcher = new faceapi.FaceMatcher(results)

// 2. Match a single face from a query image
const singleResult = await faceapi
  .detectSingleFace(queryImage1)
  .withFaceLandmarks()
  .withFaceDescriptor()

if (singleResult) {
  const bestMatch = faceMatcher.findBestMatch(singleResult.descriptor)
  console.log(bestMatch.toString())
}

  11. Load FaceAPI models

    master

    Models are accessed via the faceapi.nets object. To use them, you must provide the manifest.json and the weight shards (assets) in the same directory.

    In Browser: Use loadFromUri(path) pointing to the folder containing your model assets. In NodeJS: Use loadFromDisk(path) to load directly from the file system. From Weights: You can load from a tf.NamedTensorMap using loadFromWeightMap(weightMap) or from a Float32Array using load(weights).

    // Browser: Load from a URL path
await faceapi.nets.ssdMobilenetv1.loadFromUri('/models')

// NodeJS: Load from local disk
await faceapi.nets.ssdMobilenetv1.loadFromDisk('./models')

// Load from Float32Array (uncompressed)
const weights = new Float32Array(res.data)
net.load(weights)

  12. Install FaceAPI for NodeJS (CommonJS)

    master

    For NodeJS, FaceAPI does not bundle TFJS because of binary dependencies. You must install a TensorFlow.js backend first.

    Supported NodeJS versions: 14 through 22. (NodeJS 23+ is not supported due to TFJS incompatibility).

    Standard Installation

    Use @tensorflow/tfjs-node for CPU execution.

    GPU Accelerated Installation

    Use @tensorflow/tfjs-node-gpu if you have CUDA libraries installed.

    WASM Backend

    Use this if your platform does not support TensorFlow binary libraries.

    # Standard CPU
npm install @tensorflow/tfjs-node
npm install @vladmandic/face-api

# GPU Accelerated
npm install @tensorflow/tfjs-node-gpu
npm install @vladmandic/face-api

# WASM Backend
npm install @tensorflow/tfjs
npm install @tensorflow/tfjs-backend-wasm
npm install @vladmandic/face-api