DOCSEARCHbyAlgolia
Submit Docs

DWPose

repository·onnx·Indexed 25 days ago

https://github.com/idea-research/dwpose

An effective whole-body pose estimation framework using two-stage distillation, optimized for use with ControlNet and MMPose. The repository includes models ranging from tiny to large and incorporates various efficient architectures such as EfficientNet, MobileNetV3, and MixNet via the geffnet package, as well as integration tools for ONNX, TorchScript, and TFLite deployment.

Tokens
73.9K
Snippets
200
Records
401
Agent score
83%

What's inside DWPose

  1. Overview of DEKR (Disentangled Keypoint Regression)

    onnx
    DEKR is a bottom-up 2D human pose estimation approach. It works by simultaneously detecting all human instances in an image and regressing the offsets from the detected instance centers to the specific joints. To improve accuracy, it uses separated branches with deformable convolutional layers for different joints, allowing the model to use convolution kernels with shapes tailored to the specific features of each joint.

  3. Overview of Top-down Integral-Regression Pose Estimation

    onnx

    This project implements top-down integral-regression-based pose estimation. The process is divided into two stages:

    1. Object Detection: Identifying human bounding boxes.
    2. Pose Estimation: Estimating keypoints within the detected bounding boxes using integral regression. This method uses a differentiable integral operation to unify heatmap and joint regression, extracting keypoint coordinates directly from features within the bounding box area.

  4. Overview of the MMPose Framework

    onnx

    MMPose is an open-source PyTorch-based toolbox for pose estimation. It supports a wide range of tasks including 2D multi-person pose estimation, 2D hand pose estimation, 2D facial landmark detection, 133-keypoint full-body human pose estimation, animal keypoint detection, and clothing keypoint detection.

    The framework is organized into 8 main components:

    • apis: High-level APIs for model inference.
    • structures: Data structures such as bbox, keypoint, and PoseDataSample.
    • datasets: Support for various pose estimation datasets, including transforms for data augmentation.
    • codecs: Pose encoders (e.g., converting keypoints to heatmaps) and decoders (converting model output back to pose estimates).
    • models: Modular components including pose_estimators, data_preprocessors, backbones, necks, heads, and losses.
    • engine: Runtime components, including hooks for runtime execution.
    • evaluation: Metrics for assessing model performance.
    • visualization: Tools for visualizing keypoint skeletons, heatmaps, and other information.

  7. Overview of Top-down heatmap-based pose estimation

    onnx

    Top-down pose estimation in this project follows a two-stage process:

    1. Object Detection: Identifying human bounding boxes.
    2. Single-object Pose Estimation: Estimating keypoints within those bounding boxes.

    Instead of predicting raw coordinates, the estimator produces heatmaps representing the likelihood of keypoint locations, following the paradigm established in 'Simple Baselines for Human Pose Estimation and Tracking'.

  8. Overview of RTMPose framework

    onnx

    RTMPose is a high-performance real-time multi-person pose estimation framework built on MMPose. It is designed to bridge the gap between high-performance benchmarks and industrial requirements by optimizing paradigm, backbone networks, localization algorithms, training strategies, and deployment inference.

    Key performance metrics include:

    • RTMPose-m: Achieves 75.8% AP on COCO with 90+ FPS (Intel i7-11700 CPU) and 430+ FPS (NVIDIA GTX 1660 Ti GPU).
    • RTMPose-l: Achieves 67.0% AP on COCO-WholeBody with 130+ FPS.

  9. Overview of MMPose

    onnx

    MMPose is an open-source toolbox for pose analysis based on PyTorch and is a member of the OpenMMLab project. It supports a wide range of human pose analysis tasks, including:

    • 2D multi-person pose estimation
    • 2D hand pose estimation
    • 2D facial keypoint detection
    • 133-keypoint full-body human pose estimation
    • 3D human shape recovery
    • Clothing keypoint detection
    • Animal keypoint detection

    Key features include support for top-down and bottom-up algorithms, high precision and speed, modular design for building custom models, and support for mainstream datasets like COCO and MPII.

  10. Overview of RTMPose

    onnx

    RTMPose is a high-performance real-time multi-person pose estimation framework built on MMPose. It is designed to bridge the gap between high-performance benchmarks and industrial requirements by optimizing paradigm, backbone network, localization algorithm, training strategy, and deployment inference.

    Key performance metrics:

    • RTMPose-m: Achieves 75.8% AP on COCO with 90+ FPS on Intel i7-11700 CPU and 430+ FPS on NVIDIA GTX 1660 Ti GPU.
    • RTMPose-l: Achieves 67.0% AP on COCO-WholeBody with 130+ FPS.

  11. Overview of RTMPose toolkit

    onnx
    RTMPose is a high-performance, real-time multi-person pose estimation toolkit built on top of MMPose. It is designed for industrial applications, balancing high accuracy with low inference latency. The toolkit supports various model scales (t, s, m, l) and is optimized for deployment across multiple platforms including CPU, GPU, NVIDIA Jetson, and mobile devices (ARM) using backends like ONNX, TensorRT, ncnn, and OpenVINO.