GazeTracking

GazeTracking requires Python 3.10+. Follow one of these installation methods after cloning the repository:

Option 1: pip (Standard)

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -e .

Option 2: uv (Fast)

uv venv
uv pip install -e .

Option 3: Anaconda

conda env create --file environment.yml
conda activate GazeTracking

Option 4: Docker (Linux only)

./build_and_run.sh

Troubleshooting dlib installation: If pip attempts to compile dlib from source and fails, ensure you have cmake installed:

• macOS: brew install cmake
• Ubuntu: sudo apt install cmake build-essential

git clone https://github.com/antoinelame/GazeTracking.git
cd GazeTracking

To use the library, instantiate the GazeTracking class and pass video frames (as numpy.ndarray) to the refresh method within a loop. You can then query gaze direction, pupil positions, or retrieve an annotated frame for visualization.

import cv2
from gaze_tracking import GazeTracking

gaze = GazeTracking()
webcam = cv2.VideoCapture(0)

while True:
    _, frame = webcam.read()
    gaze.refresh(frame)

    new_frame = gaze.annotated_frame()
    text = ""

    if gaze.is_right():
        text = "Looking right"
    elif gaze.is_left():
        text = "Looking left"
    elif gaze.is_center():
        text = "Looking center"

    cv2.putText(new_frame, text, (60, 60), cv2.FONT_HERSHEY_DUPLEX, 2, (255, 0, 0), 2)
    cv2.imshow("Demo", new_frame)

    if cv2.waitKey(1) == 27:
        break

The following methods return boolean values indicating the user's eye state:

• is_left(): True if looking left.
• is_right(): True if looking right.
• is_center(): True if looking at the center.
• is_blinking(): True if the eyes are closed.

Retrieve normalized ratios (0.0 to 1.0) representing gaze direction:

• horizontal_ratio():
  • 0.0: Extreme right
  • 0.5: Center
  • 1.0: Extreme left
• vertical_ratio():
  • 0.0: Extreme top
  • 0.5: Center
  • 1.0: Extreme bottom

Attributes

• iris_frame: A numpy.ndarray representing the processed frame where the iris is isolated.
• threshold: The int value used during image processing to binarize the eye frame.
• x: The horizontal coordinate of the pupil centroid (integer).
• y: The vertical coordinate of the pupil centroid (integer).

Methods

• __init__(self, eye_frame, threshold): Initializes the detector with an eye frame and a threshold value.
• detect_iris(self, eye_frame): Processes the provided eye_frame to detect the iris and updates the x and y attributes with the centroid position.
• image_processing(eye_frame, threshold): A static method that performs bilateral filtering, erosion, and thresholding to isolate the iris from an eye frame.

Once an Eye instance is initialized, it contains several attributes representing the state of the isolated eye and its pupil:

• eye.frame: The cropped and masked numpy.ndarray containing only the eye region.
• eye.origin: A tuple (min_x, min_y) representing the top-left corner of the cropped eye frame.
• eye.center: A tuple (width / 2, height / 2) representing the center of the cropped eye frame.
• eye.pupil: An instance of the Pupil class used for pupil detection within the isolated eye frame.
• eye.blinking: A float representing the blinking ratio (width/height). A lower ratio typically indicates the eye is closed.

Returns the input frame with the pupils highlighted for visualization purposes.

frame = gaze.annotated_frame()

Pass a frame (numpy.ndarray) to the refresh method to analyze the current eye state. This must be called in a loop for video streams.

gaze.refresh(frame)

The Calibration class is used to calibrate the pupil detection algorithm by finding the optimal binarization threshold for a specific user and webcam setup. It collects threshold samples over a set number of frames (nb_frames, default is 20) to calculate an average threshold for each eye.

To use it:

1. Instantiate Calibration().
2. Call evaluate(eye_frame, side) for each eye frame captured during calibration, where side is 0 for the left eye and 1 for the right eye.
3. Check is_complete() to see if enough frames have been collected.
4. Retrieve the final threshold using threshold(side).