DOCSEARCHbyAlgolia
Submit Docs

OpenCV Zoo

repository·main·Indexed 21 days ago

https://github.com/opencv/opencv_zoo

A collection of pre-trained models optimized for the OpenCV DNN module, featuring benchmarks across x86, ARM, and RISC-V hardware platforms. The repository includes tools for performance evaluation via benchmark.py, supporting various precision levels (FP32, FP16, INT8, INT8BQ) and hardware backends such as CUDA and TIMVX (NPU). It also provides specific model implementations and demos, such as NAFNet for deblurring.

Tokens
18.8K
Snippets
70
Records
108
Agent score
77%

What's inside OpenCV Zoo

  1. Overview of CRNN Text Recognition Models

    main

    CRNN (Convolutional Recurrent Neural Network) is an end-to-end trainable neural network used for image-based sequence recognition, specifically for scene text recognition.

    Available model variants include:

    • English (EN): Detects digits (0-9) and lowercase letters (a-z).
    • Chinese (CH): Detects digits (0-9), upper/lower-case letters (a-z, A-Z), and some special characters.
    • Chinese (CN): Detects digits (0-9), upper/lower-case letters (a-z, A-Z), some Chinese characters, and some special characters.

    Models are available in standard, FP16 (half-precision), and INT8 (quantized) formats to balance accuracy and performance.

  2. Overview of OpenCV Model Zoo models and sources

    main

    The OpenCV Model Zoo provides pre-trained model weights for various computer vision tasks. As of version 4.9.0, it includes 43 model weights covering 19 tasks. Most models are available for commercial use due to their licensing.

    Key sources of models include:

    • OpenCV China Team: Developers of the YuNet face detection model.
    • OpenCV Area Chair: Contributors of SFace (face recognition) and FER (facial expression recognition) models.
    • OpenCV Partners: Includes models from Baidu PaddlePaddle (human segmentation) and watrix.ai (license plate detection).
    • OpenCV Community: Contributions from Google Summer of Code (GSoC) students covering object detection, tracking, and optical flow.

    Every model comes with ready-to-use Python and C++ demonstrations for the latest OpenCV version.

  3. Overview of NanoDet object detection model

    main

    NanoDet is a FCOS-style one-stage anchor-free object detection model that utilizes Generalized Focal Loss for both classification and regression.

    NanoDet-Plus introduces a novel label assignment strategy using an Assign Guidance Module (AGM) and a Dynamic Soft Label Assigner (DSLA) to optimize label assignment for lightweight model training.

    Model Variants:

    • The version provided is Nanodet-m-plus-1.5x_416.
    • The file object_detection_nanodet_2022nov_int8bq.onnx is a block-quantized version in int8 precision (generated with block_size=64).

  4. Overview of the MediaPipe Pose Person Detector

    main

    The MediaPipe Pose person detector is a model designed to detect upper body and full body keypoints of a person.

    Model Variants:

    • The model person_detection_mediapipe_2023mar_int8bq.onnx is a block-quantized version using int8 precision, generated with a block_size=64 using block_quantize.py.

    Model Origin: Models are typically sourced from the PINTO model zoo or converted from TFLite to ONNX using specialized tools like tflite2tensorflow and onnx-simplifier to handle MediaPipe custom densify operations.

  5. Overview of VIT tracker

    main

    VIT tracker (Vision Transformer tracker) is a high-performance model designed for real-time object tracking. It offers several advantages over nanotrack:

    • Speed: Achieves significantly higher speeds, especially in multi-threaded modes (up to 3x faster on ARM platforms).
    • Confidence Scores: Unlike nanotrack which returns a constant score, VIT tracker provides dynamic confidence values. This allows developers to programmatically determine if a target has been lost (indicated by a significant drop in the score).
    • Performance: Demonstrates higher AUC (Area Under Curve) on benchmarks like the LASOT dataset.

  6. Overview of YuNet face detection model

    main

    YuNet is a lightweight, fast, and accurate face detection model. It is designed to detect faces ranging from approximately 10x10 to 300x300 pixels.

    Model Variants

    • face_detection_yunet_2026may.onnx: The default model featuring dynamic input shape (symbolic height and width dimensions). It is compatible with the OpenCV 5.x ONNX Runtime engine. Use the environment variable OPENCV_FORCE_DNN_ENGINE=4 to enable this engine. This version allows inference at any resolution without manual resizing.
    • face_detection_yunet_2023mar.onnx: A version with a fixed input shape. While OpenCV 4.x DNN can infer on the exact shape of the input image, the ONNX Runtime engine in OpenCV 5.x requires the dynamic dimensions provided by the 2026may version for variable input sizes.
    • face_detection_yunet_2023mar_int8bq.onnx: A block-quantized version in int8 precision.

  7. Available Model Categories in OpenCV Zoo

    main

    The OpenCV Zoo provides a wide variety of models tuned for the OpenCV DNN module. Available model types include:

    • Face Analysis: Face Detection (YuNet), Face Recognition (SFace), and Facial Expression Recognition (Progressive Teacher).
    • Segmentation: Human Segmentation (PP-HumanSeg) and Image Segmentation (EfficientSAM).
    • Object Detection & Tracking: Object Detection (NanoDet, YOLOX), License Plate Detection (LPD_YuNet), and Object Tracking (VitTrack).
    • Pose & Hand Tracking: Palm Detection (MP-PalmDet), Hand Pose Estimation (MP-HandPose), Pose Estimation (MP-Pose), and Person Detection (MP-PersonDet).
    • OCR & Text: QR Code Detection and Parsing (WeChatQRCode), Text Detection (PPOCR-Det for Chinese/English), and Text Recognition (CRNN).

  8. Overview of SFace models and accuracy

    main

    SFace (Sigmoid-Constrained Hypersphere Loss for Robust Face Recognition) uses MobileFaceNet instances. The repository provides several versions of the model optimized for different precision requirements:

    • SFace: The standard model (Accuracy: 0.9940).
    • SFace block: Blockwise quantized version (Accuracy: 0.9942).
    • SFace quant: Quantized version in int8 precision (Accuracy: 0.9932).

    The face_recognition_sface_2021dec_int8bq.onnx file is a block-quantized version using block_size=64.

  10. Progressive Teacher model details and quantization

    main

    The Progressive Teacher model is a semi-supervised facial expression recognition model.

    Key Specifications:

    • Backbone: MobileFaceNet
    • Classes: 7 (angry, disgust, fearful, happy, neutral, sad, surprised)
    • Accuracy: 88.27% on RAF-DB dataset

    Available Model Files:

    • facial_expression_recognition_mobilefacenet_2022july.onnx: Standard ONNX model.
    • facial_expression_recognition_mobilefacenet_2022july_int8bq.onnx: Block-quantized version in int8 precision (generated using block_quantize.py with block_size=64).

  12. YOLOX Model Overview and Features

    main

    YOLOX is an anchor-free version of YOLO designed to bridge the gap between research and industrial applications. It provides a high-performing object detector with an optimal speed and accuracy trade-off for real-time applications.

    Key technical features:

    • Anchor-free detectors: Reduces the number of design parameters.
    • Decoupled head: Separates classification, regression, and localization to improve convergence speed.
    • SimOTA strategy: An advanced label assignment strategy that reduces training time.
    • Data augmentation: Utilizes MixUp and Mosaic to boost performance.

    Model Variants:

    • This specific implementation uses YoloX_s.
    • The file object_detection_yolox_2022nov_int8bq.onnx is a block-quantized version in int8 precision, generated with block_size=64 using tools/quantize/block_quantize.py.