Supervision Computer Vision Toolkit

Supervision is an open-source Python library by Roboflow designed for building computer vision applications.

Core Capabilities:

• Unified Detections: Provides a Detections object with converters for outputs from various models including Ultralytics, Roboflow Inference, Transformers, SAM, Detectron2, MMDetection, YOLO-NAS, PaddleDet, NCNN, Azure AI Vision, and VLM parsers.
• Annotation: Annotate images and video with bounding boxes, masks, and labels.
• Tracking: Track objects across frames using persistent IDs.
• Zone Analysis: Count and filter detections inside polygon zones.
• Dataset Management: Load and convert datasets between YOLO, COCO, and Pascal VOC formats.
• Benchmarking: Evaluate model performance using mAP and confusion matrices.

Supervision is an open-source Python library designed for building computer vision workflows. It provides a model-agnostic toolkit that allows you to decouple your model choice from your application logic.

Core capabilities include:

• Loading predictions from various models.
• Annotating images and video.
• Tracking objects.
• Counting detections within specific zones.
• Processing datasets.
• Evaluating model performance.

The library's central abstraction is the Detections API. By converting outputs from different models into this unified format, you can swap models (e.g., moving from YOLO to SAM) without rewriting your annotation, filtering, tracking, or evaluation code.

To assign persistent IDs to detected objects across video frames, use sv.ByteTrack. In your frame processing callback, pass your sv.Detections object to tracker.update_with_detections(detections). This returns updated detections that include tracker_id in their data.

import numpy as np
import supervision as sv
from ultralytics import YOLO

model = YOLO("yolov8n.pt")
tracker = sv.ByteTrack()
box_annotator = sv.BoxAnnotator()

def callback(frame: np.ndarray, _: int) -> np.ndarray:
    results = model(frame)[0]
    detections = sv.Detections.from_ultralytics(results)
    detections = tracker.update_with_detections(detections)
    return box_annotator.annotate(frame.copy(), detections=detections)

sv.process_video(
    source_path="people-walking.mp4",
    target_path="result.mp4",
    callback=callback
)

The supervision dataset API provides DetectionDataset and ClassificationDataset classes to manage computer vision datasets. These classes allow you to load, merge, split, and convert datasets across multiple formats, including:

• YOLO
• COCO
• VOC
• CreateML
• LabelMe

!!! warning

The Dataset API is currently fluid and subject to change. If using these classes in a production project, ensure you freeze the supervision version in your requirements.txt or setup.py to prevent breaking changes.

ByteTrack is model-agnostic. It accepts any Detections object containing bounding boxes, regardless of which model or converter produced them.

Key Feature: ByteTrack utilizes low-confidence detections during the association process. This helps maintain object continuity even when detections are weak or temporarily missed in certain frames.

When using CompactMask, the mask_iou_batch and mask_non_max_suppression (NMS) operations are optimized through three layers:

1. Vectorized Bbox Pre-filter: An $O(N^2)$ array operation that checks bounding box overlaps. This allows the system to skip pixel-level work for the majority of non-overlapping pairs (e.g., at 5% fill, ~96% of pairs are eliminated instantly).
2. Sub-crop Decode: Instead of comparing full frames, the system only decodes and compares the intersection region of the two overlapping crops.
3. Crop Caching: Each mask is decoded into a pixel grid at most once during the batch operation to avoid redundant RLE decoding.

This results in speedups of approximately 100x to 500x compared to dense mask IoU calculations.

Zones are defined using JSON configuration files that specify polygonal areas within the video frame. The demo provides several pre-configured zone layouts:

• horizontal-zone-config.json: Zones divided horizontally across the frame.
• multi-zone-config.json: Multiple zones with custom shapes and positions.
• quarters-zone-config.json: Splits the frame into four equal quarters.
• vertical-zone-config.json: Divides the frame into vertical zones of equal width.

The LineZone class allows you to define a line in a 2D plane and track whether detections (such as objects in a video stream) cross that line. It is useful for counting entries or exits in a specific area.

Key components:

• LineZone: The logic engine that tracks the state of detections relative to a line.
• LineZoneAnnotator: A visual tool to draw the line and the count of detections that have crossed it on an image or frame.
• LineZoneAnnotatorMulticlass: An extension of the annotator that handles multiple object classes, allowing you to visualize counts for different types of objects separately.