DOCSEARCHbyAlgolia
Submit Docs

VisDrone Multi-Drone Multi-Object Detection and Tracking

repository·main·Indexed 41 days ago

https://github.com/visdrone/multi-drone-multi-object-detection-and-tracking

A research repository providing a benchmark and the MIA-Net neural network architecture for robust multi-drone multi-target tracking. It includes implementations and configurations for various Multi-Object Tracking (MOT) methods, including ByteTrack, OC-SORT, QDTrack, Tracktor, SORT, and DeepSORT, with support for datasets such as MOT17, MOT20, LVIS, and TAO.

Tokens
12.3K
Snippets
30
Records
61
Agent score
52%

What's inside VisDrone Multi-Drone Tracking

  1. Overview of OC-SORT (Observation-Centric SORT)

    main
    OC-SORT is a Multi-Object Tracking (MOT) method designed to improve robustness against occlusions and non-linear motion. Unlike traditional motion models that assume linear motion and require continuous observations, OC-SORT emphasizes the role of 'observations' to recover tracks after they have been lost and to reduce error accumulation during lost periods. It is an online, real-time method that achieves state-of-the-art performance on datasets like MOT17, MOT20, KITTI Pedestrian Tracking, and DanceTrack without requiring appearance cues.

  2. Overview of Multi-Drone Multi-Target Tracking

    main

    This project focuses on robust multi-drone multi-target tracking, specifically addressing the challenges of identity association and target occlusion. The goal is to collaboratively detect and track targets across multiple drones and associate object identities across different drone views to overcome the limitations of single-drone tracking.

    Key components include:

    • MDMT Dataset: An occlusion-aware dataset designed for multi-drone multi-target tracking.
    • MDA (Multi-device Target Association score): An evaluation metric for cross-view target association capability.
    • MIA-Net (Multi-matching Identity Authentication network): A proposed network that uses a local-global matching algorithm to discover topological relationships of targets across drones, solving cross-drone association and complementing occluded targets.

  3. Overview of Tracktor tracking method

    main
    Tracktor is a tracking-by-detection paradigm that avoids specific training or optimization on tracking data. Instead, it exploits the bounding box regression of an object detector to predict an object's position in the subsequent frame, effectively converting a detector into a tracker. It can be extended with re-identification (ReID) and camera motion compensation to improve performance in complex scenarios like occlusions or small objects.

  4. Overview of ByteTrack Multi-Object Tracking

    main

    ByteTrack is a multi-object tracking (MOT) method designed to estimate bounding boxes and identities of objects in videos. Unlike traditional methods that only associate high-score detection boxes, ByteTrack associates every detection box. It utilizes low-score detection boxes by checking their similarity with existing tracklets to recover occluded objects and filter out background noise. This approach reduces true object missing and fragmented trajectories.

    Key performance metrics achieved on the MOT17 test set include:

    • MOTA: 80.3
    • IDF1: 77.3
    • HOTA: 63.1
    • Speed: ~30 FPS on a single V100 GPU.

  5. Overview of QDTrack (Quasi-Dense Tracking)

    main
    QDTrack is a Multiple Object Tracking (MOT) method based on Quasi-Dense Similarity Learning. Unlike traditional methods that rely on sparse ground truth matching, QDTrack uses contrastive learning by densely sampling hundreds of region proposals on image pairs. This approach allows it to be combined directly with existing detection methods (like Faster R-CNN) without requiring displacement regression or motion priors. At inference time, it utilizes a simple nearest neighbor search in a distinctive feature space.

  6. Customize components using fastreid.modeling registries

    main

    The fastreid.modeling module provides several registries that allow you to replace core components with your own customized implementations without modifying the original fastreid source code.

    To customize a component, you must identify the smallest registry that contains the logic you wish to override and register your component to that specific registry. This pattern ensures that the framework can instantiate your custom classes via configuration files rather than hardcoded imports.

  7. Organize the MDMT dataset structure

    main

    The MDMT dataset must follow this specific directory hierarchy for the project to process it correctly:

    Multi-Drone-Multi-Object-Detection-and-Tracking-main/
└── data/
    └── MDMT/
        ├── train/
        ├── val/
        └── test/
            └── sequences/
                └── img1/
                    └── [~.jpg files]
    Multi-Drone-Multi-Object-Detection-and-Tracking-main
>> data
>>> MDMT
>>>> train/
>>>> val/
>>>> test/
>>>>>> sequences
>>>>>>> img1
>>>>>>> ~.jpg

  9. Evaluate tracking performance and MDA score

    main

    Evaluation is performed in two stages: tracking performance and the MDA score.

    1. Tracking Performance

    First, convert JSON results to text format, then run the test script:

    1. python ./demo/eval/json_2_txt.py [--sequences_result] [--output_dir]
    2. python ./demo/eval/txttxt_test.py [--test_file_dir]

    2. MDA Score

    To calculate the MDA score, use: python ./demo/eval/mango_eval.py [--sequences_result]

    # Tracking performance
python ./demo/eval/json_2_txt.py [--sequences_result] [--output_dir]
python ./demo/eval/txttxt_test.py [--test_file_dir]

# MDA score
python ./demo/eval/mango_eval.py [--sequences_result]

  11. Install fast_reid using Conda

    main

    You can set up a dedicated environment for fast_reid using Conda. This method creates a Python 3.7 environment and installs the necessary PyTorch and Tensorboard packages from the official PyTorch channel before installing remaining dependencies from requirements.txt.

    conda create -n fastreid python=3.7
conda activate fastreid
conda install pytorch==1.6.0 torchvision tensorboard -c pytorch
pip install -r docs/requirements.txt