PeekingDuck

PeekingDuck nodes recognize several built-in data types for passing information through a pipeline. These types include spatial data (bboxes, masks, keypoints, zones), metadata (filename, fps, saved_video_fps), and statistical aggregations (count, cum_avg, cum_max, cum_min, density_map).

Users can also define custom data types when creating custom nodes to extend the pipeline's capabilities.

The Object Counting (Present) solution is composed of three functional stages:

1. Object Detection: Uses a model (like model.yolo) to identify bounding boxes [x1, y1, x2, y2] for specific object categories in the video feed.
2. Count Detections: The dabble.bbox_count node sums the number of bounding boxes detected for the specified object category.
3. Calculate Statistics: The dabble.statistics node processes the counts from each frame to calculate cumulative metrics: cum_avg (cumulative average), cum_max (cumulative maximum), and cum_min (cumulative minimum).

This is distinct from advanced counting tasks like tracking objects over time or counting within specific zones.

Zone counting identifies objects within specific areas of a video feed by following these steps:

1. Object Detection: An object detection model (like model.yolo) identifies bounding boxes $[x_1, y_1, x_2, y_2]$ for target objects.
2. Bottom Midpoint Calculation: To determine if an object is

x_bm = (x1 + x2) / 2
y_bm = y2

If you are running on resource-constrained hardware like a Raspberry Pi, you can replace pose estimation with object detection. This is often more efficient as lightweight object detectors typically perform better than lightweight pose estimation models.

Implementation Steps:

1. Replace the model.posenet node with an object detection model node.
2. Replace the dabble.keypoints_to_3d_loc node with the dabble.bbox_to_3d_loc node.

Trade-offs:

• Accuracy: Distance estimation will be less accurate because the bounding box height changes if a person sits or bends, whereas pose estimation relies on skeletal points.

When building custom object detection model nodes in PeekingDuck, you should adhere to the project's standard data type conventions to ensure compatibility with existing nodes (like input.visual for reading sources or draw.bbox for visualization).

Input Requirements:

• Receive the img data type.
• The img data must be in BGR format.

Output Requirements: Your model should produce the following data types:

• bboxes: A NumPy array of the expected shape.
• bbox_labels: Labels associated with the bounding boxes.
• bbox_scores: Confidence scores for the detections.

The privacy protection solution follows a two-step process:

1. Segmentation: An instance segmentation model (like model.mask_rcnn) identifies the masks for sensitive objects such as persons, laptops, or computer screens. The mask is a single-channel binary image where 1 indicates the presence of the object class and 0 indicates absence.
2. De-identification: The draw.instance_mask node applies a Gaussian blur or a mosaic (pixelation) effect to the pixels where the mask value is 1.

When implementing face detection, choose between these two nodes based on your requirements:

• model.yolo_face: Use this if you need to classify whether a face is wearing a mask or not. It uses YOLOv4/YOLOv4-tiny to identify bounding boxes and assign classes (no_mask = 0 or mask = 1).
• model.mtcnn: Use this if you only need to detect the presence of human faces without the need for mask classification.

The people counting over time use case follows a three-step process:

1. Human Detection: Uses the model.jde node to detect persons and provide bounding box coordinates [x1, y1, x2, y2].
2. Appearance Embedding Tracking: Uses a metric learning algorithm with triplet loss, the Hungarian algorithm for assignment, and a Kalman filter for trajectory smoothing. This assigns a unique ID to each detected person.
3. Incrementing the Count: New unique persons are assigned monotonically increasing integer IDs starting from 0. The total count of unique persons is calculated as the cumulative maximum (cum_max) of these IDs.