DOCSEARCHbyAlgolia
Submit Docs

face-api.js

repository·master·Indexed 12 days ago

https://github.com/justadudewhohacks/face-api.js

A JavaScript API for face detection and face recognition in the browser and Node.js, built on top of TensorFlow.js. Version 0.22.2 supports landmark detection, expression recognition, age and gender estimation, and face descriptors for identity matching using the FaceMatcher class. It includes multiple model types such as ssdMobilenetv1 and tinyFaceDetector, and provides utilities for drawing detection results on a canvas.

Tokens
13.9K
Snippets
52
Records
71
Agent score
94%

What's inside face-api.js

  1. Understand the Face Expression Recognition Model

    master

    The face expression recognition model is designed to be lightweight and fast. It uses depthwise separable convolutions and densely connected blocks to provide reasonable accuracy with a small footprint.

    Model Details:

    • Size: Approximately 310 KB.
    • Note: Accuracy may decrease if the subject is wearing glasses.

  2. Compose multiple face detection tasks

    master

    The API allows you to chain multiple analysis tasks together. The order typically follows: detect -> withFaceLandmarks -> withFaceExpressions -> withAgeAndGender -> withFaceDescriptors.

    // Example of a fully composed task for all faces
await faceapi.detectAllFaces(input)
  .withFaceLandmarks()
  .withFaceExpressions()
  .withAgeAndGender()
  .withFaceDescriptors()

  3. Understand the Age and Gender Recognition Model

    master

    The age and gender recognition model is a multitask network consisting of a feature extraction layer, an age regression layer, and a gender classifier. It uses a feature extractor architecture similar to Xception.

    Performance Metrics:

    • Total Mean Age Error (MAE): 4.54
    • Total Gender Accuracy: 95%

    Model Details:

    • Size: Approximately 420 KB.
    • Training Data: Trained on multiple databases including UTK, FGNET, Chalearn, Wiki, IMDB*, CACD*, MegaAge, and MegaAge-Asian.

  4. Supported input types for face-api.js

    master

    The API accepts several types of input for face detection and analysis. You can provide an HTML <img>, <video>, or <canvas> element, or simply the id of that element as a string.

    const input = document.getElementById('myImg')
// const input = document.getElementById('myVideo')
// const input = document.getElementById('myCanvas')
// or simply:
// const input = 'myImg'

  5. Understand the Face Recognition Model

    master

    The face recognition model uses a ResNet-34-like architecture to compute a face descriptor, which is a feature vector containing 128 values. This descriptor represents the unique characteristics of a face.

    Key Capabilities:

    • Generalization: The model is not limited to the faces used during training; it can recognize arbitrary people (including yourself).
    • Similarity Comparison: You can determine if two faces are the same by comparing their 128-value descriptors using Euclidean distance or other classifiers.
    • Accuracy: It achieves 99.38% prediction accuracy on the LFW (Labeled Faces in the Wild) benchmark.

    Model Details:

    • Name: face_recognition_model
    • Size: Approximately 6.2 MB (quantized).
    // Conceptual usage: 
// 1. Compute descriptor for Face A
// 2. Compute descriptor for Face B
// 3. Compare descriptors using Euclidean distance
const distance = euclideanDistance(descriptorA, descriptorB);

  6. Display detection results on a canvas

    master

    To visualize face detection results, you must first prepare an overlay canvas that matches the dimensions of your input image or video. Use faceapi.matchDimensions to sync the canvas size with the input.

    Once detections are made, use faceapi.resizeResults to scale the detection coordinates to match the displayed dimensions of the canvas before drawing.

    Common drawing tasks include:

    • Bounding Boxes: Use faceapi.draw.drawDetections(canvas, resizedResults).
    • Landmarks: Use faceapi.draw.drawFaceLandmarks(canvas, resizedResults).
    • Expressions: Use faceapi.draw.drawFaceExpressions(canvas, resizedResults, minProbability) where minProbability is the threshold for displaying an expression.
    const displaySize = { width: input.width, height: input.height }
// resize the overlay canvas to the input dimensions
const canvas = document.getElementById('overlay')
faceapi.matchDimensions(canvas, displaySize)

/* Example: Displaying detections and landmarks */
const detectionsWithLandmarks = await faceapi
  .detectAllFaces(input)
  .withFaceLandmarks()

// resize the detected boxes and landmarks to match the display size
const resizedResults = faceapi.resizeResults(detectionsWithLandmarks, displaySize)

// draw to canvas
faceapi.draw.drawDetections(canvas, resizedResults)
faceapi.draw.drawFaceLandmarks(canvas, resizedResults)

  8. Load face-api.js models

    master

    Models are accessed via the faceapi.nets object. To use a model, you must provide the path to the directory containing the model's manifest.json and its weight shards. The manifest and shards must be in the same directory.

    Available model types include:

    • ssdMobilenetv1
    • tinyFaceDetector
    • tinyYolov2
    • faceLandmark68Net
    • faceLandmark68TinyNet
    • faceRecognitionNet
    • faceExpressionNet
    • ageGenderNet
    // Loading from a URI (Browser/Web)
await faceapi.nets.ssdMobilenetv1.loadFromUri('/models')

// Loading from disk (Node.js)
await faceapi.nets.ssdMobilenetv1.loadFromDisk('./models')

// Loading from a weight map
await faceapi.nets.ssdMobilenetv1.loadFromWeightMap(weightMap)

// Creating a custom instance
const net = new faceapi.SsdMobilenetv1()
await net.loadFromUri('/models')

  9. Configure the Node.js environment for face-api.js

    master

    When running in Node.js, you must monkey patch the faceapi.env to provide implementations for Canvas, Image, and ImageData using the canvas package. This allows the API to function similarly to the browser.

    // import nodejs bindings to native tensorflow
import '@tensorflow/tfjs-node';

// implements nodejs wrappers for HTMLCanvasElement, HTMLImageElement, ImageData
import * as canvas from 'canvas';
import * as faceapi from 'face-api.js';

// patch nodejs environment
const { Canvas, Image, ImageData } = canvas
faceapi.env.monkeyPatch({ Canvas, Image, ImageData })

  10. Perform face recognition using FaceMatcher

    master

    Face recognition is achieved by comparing a query face descriptor against a set of reference descriptors using the FaceMatcher class.

    1. Initialize Reference Data: Detect faces in a reference image and extract their descriptors.
    2. Create FaceMatcher: Pass the detection results to new faceapi.FaceMatcher(results).
    3. Match Query: Use faceMatcher.findBestMatch(descriptor) on a new face descriptor to find the closest match.
    // 1. Setup FaceMatcher from a reference image
const results = await faceapi.detectAllFaces(referenceImage).withFaceLandmarks().withFaceDescriptors()
const faceMatcher = new faceapi.FaceMatcher(results)

// 2. Recognize a single face in a query image
const singleResult = await faceapi.detectSingleFace(queryImage1).withFaceLandmarks().withFaceDescriptor()
if (singleResult) {
  const bestMatch = faceMatcher.findBestMatch(singleResult.descriptor)
  console.log(bestMatch.toString())
}

  11. Install face-api.js for Node.js

    master

    To use face-api.js in a Node.js environment, it is recommended to install canvas to polyfill browser-specific elements (like HTMLCanvasElement and HTMLImageElement) and @tensorflow/tfjs-node to significantly improve performance by using native C++ bindings.

    npm i face-api.js canvas @tensorflow/tfjs-node

  12. Run face-api.js examples

    master

    To run the provided examples from a local clone of the repository:

    Browser Examples:

    1. Navigate to examples/examples-browser
    2. Run npm i
    3. Run npm start
    4. Open http://localhost:3000/

    Node.js Examples:

    1. Navigate to examples/examples-nodejs
    2. Run npm i
    3. Run using ts-node: ts-node faceDetection.ts
    4. Or compile and run with node: tsc faceDetection.ts && node faceDetection.js
    # Browser setup
cd face-api.js/examples/examples-browser
npm i
npm start

# Node.js setup
cd face-api.js/examples/examples-nodejs
npm i
ts-node faceDetection.ts