DOCSEARCHbyAlgolia
Submit Docs

Human AI Library

repository·main·Indexed 25 days ago

https://github.com/vladmandic/human

An AI-powered library for high-performance 3D face, body, hand, and iris tracking, as well as gesture and emotion recognition. It supports browser and NodeJS environments with features including FaceID validation, anti-spoofing, liveness detection, and multi-threaded face matching. The library provides tools for face descriptor analysis using models like faceres.json, insightface, and mobilefacenet, and includes a tracker utility for moving objects.

Tokens
14.2K
Snippets
20
Records
95
Agent score
85%

What's inside @vladmandic/human

  1. Overview of Human Library capabilities

    main

    Human is an AI-powered library providing a unified API for various computer vision tasks. Key capabilities include:

    • Face Analysis: 3D Face Detection & Rotation Tracking, Face Description & Recognition, Iris Analysis, Age, Gender, Emotion Prediction, and Gaze Tracking.
    • Body & Hand Tracking: Body Pose Tracking, 3D Hand & Finger Tracking, and Body Segmentation.
    • Interaction: Gesture Recognition.

    The library is designed to be efficient by using an attention pipeline to execute related models only when needed and employing temporal interpolation for smooth results.

  3. Available NodeJS Demo Implementations

    main

    The vladmandic/human repository provides several specialized demo implementations for NodeJS to handle different input types and processing requirements:

    • node: Processes images from local files, folders, or URLs using native methods for loading and decoding without external dependencies.
    • node-canvas: Processes images from files or URLs and draws detection results to a new image file using the node-canvas library.
    • node-video: Processes video input (files, streams, or devices like webcams) using ffmpeg to decode frames for the Human library.
    • node-webcam: Uses fswebcam to capture screenshots from a webcam at regular intervals for processing.
    • node-event: Demonstrates how to use Human eventing to receive notifications during processing.
    • node-similarity: Compares two input images to determine the similarity of detected faces.
    • process-folder: A utility used to process all images within an input folder and generate output images (internally used for sample galleries).

  6. Run Human Multithreading Demos in NodeJS

    main

    You can run parallel execution of the human module in NodeJS using a multi-process approach. This is achieved by dispatching tasks to a pool of pre-created worker processes.

    The demo uses nodejs/node-multiprocess.js and nodejs/node-multiprocess-worker.js which utilize CommonJS modules to start n child worker processes for parallel execution.

    node demo/nodejs/node-multiprocess.js

  7. Use the TypeScript Browser Demo as a Quick-Start Guide

    main
    The TypeScript demo provides a simple application designed to serve as a quick-start guide for using the Human library in browser environments. The project structure relies on index.ts being compiled into index.js, which is then loaded by index.html.

  8. Explore Human Library demos

    main

    The project provides numerous live demos to showcase capabilities and implementation patterns:

    Browser Demos

    • Full Demo: Showcases all Human capabilities (Live).
    • Simple Demo: TypeScript-based webcam processing (Live).
    • Face Match: Extracts faces and matches them against a database (Live).
    • Multi-thread: Uses Web Workers for high performance (Live).
    • Framework Integrations: Includes demos for NextJS, ElectronJS, BabylonJS, and Three.js.

    NodeJS Demos

    Note: NodeJS demos may require extra dependencies for input decoding (e.g., ffmpeg, fswebcam, node-canvas).

    • Main: Process images from files, folders, or URLs.
    • Canvas: Draw results to new image files using node-canvas.
    • Video: Process video input using ffmpeg.
    • Events: Showcase usage of Human eventing for processing notifications.

  10. Use WebRTC as a Camera Source

    main

    The Human demo supports WebRTC streams as an input source. To use this, you must provide a WebRTC server that delivers a compatible media track (such as an H.264 video track).

    For an implementation that connects to IP Security cameras via RTSP and transcodes them to WebRTC, refer to the stream-rtsp project.

  11. Understand the FaceID Workflow

    main

    The faceid feature performs a multi-stage validation process on webcam input before attempting a face match. The workflow is as follows:

    1. Webcam Initialization: Starts the webcam stream.
    2. Validation Phase: Waits for the input video to meet specific criteria or reach a timeout. Criteria include:
      • Number of people detected.
      • Face size.
      • Face and gaze direction.
      • Detection scores.
      • Blink detection (including temporal checks for blink speed to verify live input).
      • Optional antispoofing module execution.
      • Optional liveness module execution.
    3. Matching Phase: Runs the validated face against a database of registered faces stored in client-side IndexDB and presents the best match with associated scores.

  12. Quick Start via CDN

    main

    To use the IIFE version of Human directly in an HTML file, load it from a CDN. You can use jsdelivr, unpkg, or cdnjs.

    <!DOCTYPE HTML>
<script src="https://cdn.jsdelivr.net/npm/@vladmandic/human/dist/human.js"></script>
<script src="https://unpkg.dev/@vladmandic/human/dist/human.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/human/3.0.0/human.js"></script>