Deep SORT Realtime

The tracker supports polygon detections by converting them to bounding rectangles.

Setup:

1. When instantiating DeepSort, set the polygon=True argument.
2. Provide detections as polygons (format details in DeepSort.update_tracks docstring).

Behavior:

• Tracking is performed on the bounding rectangle of the polygon.
• If embedding is enabled, the embedder works on the crop around the bounding rectangle, but the area not covered by the polygon is masked away.
• The original polygon coordinates are passed to the associated track via the supplementary info mechanism.

You can install the package via PyPI using pip, or by cloning the repository and installing it locally.

Via PyPI:

pip3 install deep-sort-realtime

From source:

cd deep_sort_realtime && pip3 install .

You can also download the .whl file from the project's releases page.

To use the tracker, instantiate DeepSort and call update_tracks with a list of detections.

Detection Format: bbs (bounding boxes) should be a list of tuples, where each tuple contains:

1. [left, top, w, h] (the bounding box coordinates)
2. confidence (float)
3. detection_class (class identifier)

Example:

from deep_sort_realtime.deepsort_tracker import DeepSort

tracker = DeepSort(max_age=5)
# bbs expected to be a list of detections: ([left,top,w,h], confidence, detection_class)
bbs = object_detector.detect(frame) 
tracks = tracker.update_tracks(bbs, frame=frame)

for track in tracks:
    if not track.is_confirmed():
        continue
    track_id = track.track_id
    ltrb = track.to_ltrb()

If you have your own feature extraction logic, you can pass pre-computed embeddings directly to update_tracks. This avoids the need to pass the frame argument to the tracker.

Example:

from deep_sort_realtime.deepsort_tracker import DeepSort

tracker = DeepSort(max_age=5)
bbs = object_detector.detect(frame) # your own object detection
object_chips = chipper(frame, bbs) # your own logic to crop frame based on bbox values
embeds = embedder(object_chips) # your own embedder to take in the cropped object chips, and output feature vectors

# Pass embeds directly; frame is no longer required
tracks = tracker.update_tracks(bbs, embeds=embeds)

for track in tracks:
    if not track.is_confirmed():
        continue
    track_id = track.track_id
    ltrb = track.to_ltrb()

You can pass extra data (like instance segmentation masks) from a detection to its associated track using the others argument in DeepSort.update_tracks. This argument expects a list of the same length as raw_detections.

To retrieve this data from a track, use the Track.get_det_supplementary method.

By default, Track.to_* methods return Kalman predicted bounding box values. To retrieve the bounding box values from the original detection associated with the track in the current round, use the orig=True argument.

• orig=True: Returns the original detection's bounding box if the track is associated with a detection this round; otherwise returns Kalman predicted values.
• orig_strict=True: (Only active when orig=True) Returns None if there is no original detection associated with this track in the current frame; otherwise returns the original detection's bounding box.