OC-SORT (Observation-Centric SORT)

repository·master·Indexed 22 days ago

A pure motion-model-based multi-object tracker designed for robustness in crowded scenes and non-linear motion. The repository provides a C++ library implementation requiring Eigen3 and OpenCV, as well as deployment guides for ncnn and ONNXRuntime, including PyTorch model conversion and ncnn parameter optimization for Focus layer support.

Tokens: 9.6K
Snippets: 29
Records: 40
Agent score: 78%

What's inside oc_sort

Train OC-SORT models
master
OC-SORT can be used with existing detectors without training. However, if you wish to train your own detector, follow these guidelines. First, download the COCO-pretrained YOLOX weights and place them in <OCSORT_HOME>/pretrained.
Training Commands
- Ablation model (MOT17 half train and CrowdHuman):
```
python3 tools/train.py -f exps/example/mot/yolox_x_ablation.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
```
- MOT17 test model (MOT17 train, CrowdHuman, Cityperson and ETHZ):
```
python3 tools/train.py -f exps/example/mot/yolox_x_mix_det.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
```
- MOT20 test model (MOT20 train, CrowdHuman): Note: For MOT20, you must manually uncomment specific code lines in the YOLOX source (related to box clipping in data_augment, mosaicdetection.py, and boxes.py) before running.
```
python3 tools/train.py -f exps/example/mot/yolox_x_mix_mot20_ch.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
```
- DanceTrack train set:
```
python3 tools/train.py -f exps/example/dancetrack/yolox_x.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
```
- Custom dataset:
1. Prepare your dataset in COCO format.
2. Create an Exp file for your dataset (refer to the CrowdHuman example).
3. Modify get_data_loader() and get_eval_loader() in your Exp file.
4. Run training:
```
python3 tools/train.py -f exps/example/mot/your_exp_file.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
```
https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md
Create mixed training sets for MOT17 and MOT20
master
If you are training specifically for MOT17 or MOT20, you can create mixed training sets using the ablation or specific dataset scripts.
```
# build mixed training sets for MOT17 and MOT20 
python3 tools/mix_data_{ablation/mot17/mot20}.py
```
https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md
Convert a PyTorch model to TensorRT
master
You can convert a PyTorch model (e.g., bytetrack_s_mot17) into a TensorRT model using the tools/trt.py script. You need to provide the experiment configuration file and the pretrained checkpoint.
```
cd <ByteTrack_HOME>
python3 tools/trt.py -f exps/example/mot/yolox_s_mix_det.py -c pretrained/bytetrack_s_mot17.pth.tar
```
https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/python/README.md
Perform offline interpolation on tracking results
master
While OC-SORT is designed for online tracking, you can use offline interpolation to improve trajectories.
Linear Interpolation
Use linear interpolation over existing tracking results:
```
python3 tools/interpolation.py $result_path $save_path
```
Gaussian Process Regression (GPR) Interpolation
You can also attempt Gaussian Process Regression, which works upon existing linear interpolation results:
```
python3 tools/gp_interpolation.py $raw_results_path $linear_interp_path $save_path
```
https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md
Export OC-SORT model to ONNX format
master
To use OC-SORT with ONNX Runtime, you must first convert the PyTorch model to an ONNX checkpoint using the export_onnx.py script. You can specify the output name and the experiment file (exp) that defines the model architecture.
Note: For faster inference in production, consider using a smaller model configuration.
```
python deploy/scripts/export_onnx.py --output-name ocsort.onnx -f exps/example/mot/yolox_x_mix_det.py -c pretrained/bytetrack_x_mot17.pth.tar
```
https://github.com/noahcao/oc_sort/blob/master/docs/DEPLOY.md
Prepare the serialized TensorRT engine file
master
The C++ demo requires a serialized TensorRT engine file. You must first follow the TensorRT Python demo to convert and save the engine file. The resulting file model_trt.engine should be located in the YOLOX_output directory.
https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/cpp/README.md

Evaluate OC-SORT on various datasets

master

Use the following commands to evaluate OC-SORT on different benchmark datasets. Ensure you have the correct pretrained weights and experiment files.

Evaluation Commands

DanceTrack (Val/Test):

# Val
python tools/run_ocsort_dance.py -f exps/example/mot/yolox_dancetrack_val.py -c pretrained/bytetrack_dance_model.pth.tar -b 1 -d 1 --fp16 --fuse --expn $exp_name

# Test
python tools/run_ocsort_dance.py -f exps/example/mot/yolox_dancetrack_test.py -c pretrained/bytetrack_dance_model.pth.tar -b 1 -d 1 --fp16 --fuse --test --expn $exp_name

MOT17 (Half Val/Test):

# Half Val
python3 tools/run_ocsort.py -f exps/example/mot/yolox_x_ablation.py -c pretrained/bytetrack_ablation.pth.tar -b 1 -d 1 --fp16 --fuse --expn $exp_name

# Test
python3 tools/run_ocsort.py -f exps/example/mot/yolox_x_mix_det.py -c pretrained/bytetrack_x_mot17.pth.tar -b 1 -d 1 --fp16 --fuse --expn $exp_name

MOT20 (Test):

python3 tools/run_ocsort.py -f exps/example/mot/yolox_x_mix_mot20_ch.py -c pretrained/bytetrack_x_mot20.tar -b 1 -d 1 --fp16 --fuse --track_thresh 0.4 --mot20 --expn $exp_name

KITTI (Test) (Using PermaTrack detection results):

python tools/run_ocsort_public.py  --hp --out_path kitti_test --dataset kitti --raw_results_path exps/permatrack_kitti_test

HeadTrack21 (CroHD):

python tools/run_ocsort_public.py  --raw_results_path datasets/headtrack/$tracker --hp --dataset headtrack --out_path out_headtrack_results

Compile the OC-SORT C++ Library with CMake

master

To build the OC-SORT C++ library, you need a C++17 compatible compiler and the Eigen3 and OpenCV libraries installed on your system.

Prerequisites

CMake: Version 3.10 or higher.
Eigen3: Required for matrix operations.
OpenCV: Required for image processing and video handling.

Build Steps

Configure Paths: In your CMakeLists.txt, set the paths to your local Eigen3 and OpenCV installations using SET(Eigen3_DIR ...) and set(OpenCV_DIR ...).
Generate Build Files: Use the CMake GUI or command line to configure the project.
Compile: Build the project to generate the shared library (e.g., libocsort.dll on Windows or .so on Linux).

Example CMakeLists.txt Structure

cmake_minimum_required(VERSION 3.10)
project(libocsort)

set(CMAKE_CXX_STANDARD 17)

# Linker external library
SET(Eigen3_DIR "C:/eigen-3.4.0/build")
find_package(Eigen3 REQUIRED)
set(OpenCV_DIR "C:/opencv")
find_package(OpenCV REQUIRED)

# Library definition
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
file(GLOB SRC_LIST src/*.cpp)
add_library(${PROJECT_NAME} SHARED ${SRC_LIST})
target_include_directories(${PROJECT_NAME} PUBLIC include)
target_link_libraries(${PROJECT_NAME} Eigen3::Eigen)

# Executable for testing (e.g., with YOLO inference)
add_executable(test mutilthread.cpp <inference_header> <inference_source>)
target_link_libraries(test PUBLIC Eigen3::Eigen ${PROJECT_NAME} ${OpenCV_LIBS})

cmake_minimum_required(VERSION 3.10)
project(libocsort)

set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /O2")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_RELEASE} /O2") 
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS_RELEASE} /O2")

set(CMAKE_CXX_STANDARD 17)

# Linker external library
SET(Eigen3_DIR "C:/eigen-3.4.0/build")
find_package(Eigen3 REQUIRED)
set(OpenCV_DIR "C:/opencv")
find_package(OpenCV REQUIRED)

# add_subdirectory(src)
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
file(GLOB SRC_LIST src/*.cpp)

add_library(${PROJECT_NAME} SHARED ${SRC_LIST})
target_include_directories(${PROJECT_NAME} PUBLIC include)
target_link_libraries(${PROJECT_NAME} Eigen3::Eigen)

# note：test with yolo inference
add_executable(test mutilthread.cpp <inference_header> <inference_source>)
target_link_libraries(test PUBLIC Eigen3::Eigen ${PROJECT_NAME} ${OpenCV_LIBS})

Deploy OC-SORT with TensorRT (Python)
master
To use TensorRT for accelerated inference in Python, follow these steps:
1. Prerequisites: Install TensorRT (Version 7 is recommended) following the NVIDIA Installation Guide and install torch2trt.
2. Model Conversion: Convert the model using deploy/scripts/trt.py. You will need a checkpoint such as bytetrack_s_mot17.pth.tar from the ByteTrack model zoo.
3. Run Demo: Use tools/demo_track.py with the --trt flag to run inference on a video and save the results.
Note: C++ support for TensorRT is currently unvalidated in this repository. For C++ TensorRT adaptation, refer to the ByteTrack deployment guidance.
```
# 1. Convert Model
python3 deploy/scripts/trt.py -f exps/example/mot/yolox_s_mix_det.py -c pretrained/bytetrack_s_mot17.pth.tar

# 2. Run on a demo video
python3 tools/demo_track.py video -f exps/example/mot/yolox_s_mix_det.py --trt --save_result
```
https://github.com/noahcao/oc_sort/blob/master/docs/DEPLOY.md

Prepare datasets for OC-SORT

master

To use OC-SORT, you must download the required datasets and organize them under <OCSORT_HOME>/datasets following a specific directory structure. Supported datasets include MOT17, MOT20, CrowdHuman, Cityperson, ETHZ, and DanceTrack.

Directory Structure Example

datasets
|——————mot
|        └——————train
|        └——————test
└——————crowdhuman
|        └——————Crowdhuman_train
|        └——————Crowdhuman_val
|        └——————annotation_train.odgt
|        └——————annotation_val.odgt
└——————MOT20
|        └——————train
|        └——————test
└——————Cityscapes
|        └——————images
|        └——————labels_with_ids
└——————ETHZ
|        └——————eth01
|        └——————...
|        └——————eth07
└——————dancetrack        
         └——————train
         └——————val
         └——————test

# No specific command, follow the directory structure above.

Deploy OC-SORT with ncnn
master
To deploy OC-SORT using the ncnn framework, follow the official ByteTrack ncnn deployment guidelines.
https://github.com/noahcao/oc_sort/blob/master/docs/DEPLOY.md
Install OC-SORT on the host machine
master
To install OC-SORT directly on your host machine, ensure you have Python 3.8 installed. The process involves cloning the repository, installing dependencies via requirements.txt, and running the setup script in development mode. You must also install pycocotools and several additional packages like cython_bbox, pandas, and xmltodict.
```
# Step 1: Install OC-SORT
git clone https://github.com/noahcao/OC_SORT.git
cd OC_SORT
pip3 install -r requirements.txt
python3 setup.py develop

# Step 2: Install pycocotools
pip3 install cython; pip3 install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'

# Step 3: Install other dependencies
pip3 install cython_bbox pandas xmltodict
```
https://github.com/noahcao/oc_sort/blob/master/docs/INSTALL.md

OC-SORT (Observation-Centric SORT)

What's inside oc_sort

Training Commands

Linear Interpolation

Gaussian Process Regression (GPR) Interpolation

Evaluation Commands

Prerequisites

Build Steps

Example CMakeLists.txt Structure

Directory Structure Example