clmtrackr Documentation

repository·dev·Indexed 27 days ago

https://github.com/auduno/clmtrackr

A JavaScript library (v1.1.2) for precise tracking of facial features in videos or images using Constrained Local Models (CLM) and regularized landmark mean-shift. It provides functionality to initialize trackers, retrieve facial feature coordinates via getCurrentPosition(), and render models or debugging data onto HTML canvas elements.

Tokens
1.3K
Snippets
4
Records
9
Agent score
42%

What's inside clmtrackr

  1. Install clmtrackr via script tag

    dev

    To use clmtrackr in a web project, download the minified clmtrackr.js file and include it in your HTML using a <script> tag.

    /* clmtrackr libraries */
    <script src="js/clmtrackr.js"></script>
  2. Get facial feature positions with getCurrentPosition()

    dev

    Use the getCurrentPosition() method on your tracker instance to retrieve the coordinate positions of the tracked facial features. The method returns an array of coordinates in the format [[x_0, y_0], [x_1, y_1], ... ].

    <script type="text/javascript">
      function positionLoop() {
        requestAnimationFrame(positionLoop);
        var positions = ctracker.getCurrentPosition();
        // positions = [[x_0, y_0], [x_1,y_1], ... ]
        // do something with the positions ...
      }
      positionLoop();
    </script>
  3. Draw the facial model on a canvas with draw()

    dev

    The draw(canvas) method allows you to render the tracked facial model directly onto an HTML <canvas> element. It is typically used within an animation loop (e.g., requestAnimationFrame) to provide real-time visual feedback.

    <canvas id="drawCanvas" width="400" height="300"></canvas>
    <script type="text/javascript">
      var canvasInput = document.getElementById('drawCanvas');
      var cc = canvasInput.getContext('2d');
      function drawLoop() {
        requestAnimationFrame(drawLoop);
        cc.clearRect(0, 0, canvasInput.width, canvasInput.height);
        ctracker.draw(canvasInput);
      }
      drawLoop();
    </script>
  4. Initialize and start the clmtrackr tracker

    dev

    To start tracking a face in a video element, instantiate clm.tracker(), call .init() to initialize it, and then call .start(videoElement) where videoElement is a reference to your HTML <video> element.

    <video id="inputVideo" width="400" height="300" autoplay loop>
      <source src="./media/somevideo.ogv" type="video/ogg"/>
    </video>
    <script type="text/javascript">
      var videoInput = document.getElementById('inputVideo');
      
      var ctracker = new clm.tracker();
      ctracker.init();
      ctracker.start(videoInput);
    </script>
  5. Draw detection results with drawDetection

    dev

    Use drawDetection to visualize a face detection result on a canvas. This function clears the canvas and draws the bounding box, eye filter areas, nose filter area, and facial points.

    Parameters:

    • ctx: The Canvas 2D context.
    • bbox: An object containing { x, y, width, height } representing the bounding box.
    • facePoints: An array of coordinates representing facial landmarks.
  6. Visualize image patches with drawPatches

    dev

    Use drawPatches to draw multiple image patches onto a canvas. This is useful for visualizing how the model sees different parts of the face.

    Parameters:

    • ctx: The Canvas 2D context.
    • patches: An array of patch data arrays.
    • patchSize: The size (width and height) of each patch.
    • patchPositions: An array of [x, y] coordinates representing where each patch should be drawn.
    • mapFunction (optional): A function to transform the patch data before drawing.
    • subset (optional): An array of indices specifying which patches to draw.
  7. Draw facial points with drawFacialPoints

    dev

    Use drawFacialPoints to draw specific facial landmarks (specifically the left eye, right eye, and nose) onto a canvas context. You can optionally apply a transformation to the points.

    Parameters:

    • ctx: The Canvas 2D context.
    • facePoints: An array of coordinates.
    • transformParams (optional): An array [translateX, translateY, scaling, rotation] used to transform the points before drawing.

    Note: The function sets the fill style to rgb(200,10,100) when transformations are applied.

  8. Visualize pixel data with drawData

    dev

    Use drawData to draw raw pixel data onto a canvas. This is primarily a debugging utility for visualizing grayscale or intensity data.

    Parameters:

    • canvasContext: The Canvas 2D context.
    • data: The array of pixel values.
    • width: Width of the data block.
    • height: Height of the data block.
    • transposed: Boolean indicating if the data is transposed.
    • drawX: The X coordinate on the canvas to start drawing.
    • drawY: The Y coordinate on the canvas to start drawing.