DOCSEARCHbyAlgolia
Submit Docs

MMPose Documentation

repository·main·Indexed 26 days ago

https://github.com/open-mmlab/mmpose

An open-source PyTorch-based toolbox for pose estimation tasks, including 2D/3D human pose, hand pose, face landmarks, and animal pose estimation. It supports top-down and bottom-up architectural approaches, featuring high-performance frameworks like RTMPose for real-time multi-person pose estimation and RTMO for one-stage inference. The library provides a variety of models (e.g., HRNet, ResNet, CSPNeXt) and supports multiple datasets including COCO, MPII, CrowdPose, and various animal-specific datasets.

Tokens
74.2K
Snippets
171
Records
418
Agent score
92%

What's inside MMPose

  1. Overview of RTMPose for real-time 2D pose estimation

    main
    RTMPose is a high-performance real-time multi-person pose estimation framework built on MMPose. It is designed to bridge the gap between high-performance benchmarks and industrial requirements by optimizing paradigm, backbone networks, localization algorithms, training strategies, and deployment inference. It is suitable for applications requiring high FPS on both CPUs (e.g., Intel i7) and GPUs (e.g., NVIDIA GTX series), as well as mobile devices.

  2. Overview of Top-down integral-regression-based pose estimation

    main

    Top-down integral-regression-based pose estimation methods operate in two stages:

    1. Object Detection: Identifying human bounding boxes.
    2. Single-object Pose Estimation: Estimating keypoints within the detected bounding boxes.

    Integral regression methods unify heatmap and joint regression through a differentiable integral operation. This allows the model to obtain keypoint coordinates directly from features extracted from the bounding box area, following the paradigm introduced in Integral Human Pose Regression.

  3. Overview of DEKR (Disentangled Keypoint Regression)

    main
    DEKR is a bottom-up 2D human pose estimation approach. It works by simultaneously detecting all person instances and regressing the offsets from the instance centers to the joints. To improve accuracy, it uses separated branches with deformable convolutional layers for different joints, allowing the model to use convolution kernels of different shapes tailored to each specific joint's feature extraction requirements.

  4. Overview of RTMPose for 2D Pose Estimation

    main
    RTMPose is a high-performance real-time multi-person pose estimation framework built on MMPose. It is designed to bridge the gap between high-performance benchmarks and industrial requirements by optimizing paradigm, backbone networks, localization algorithms, training strategies, and deployment inference. It is optimized for low latency and low parameter counts, making it suitable for CPU, GPU, and mobile device deployment.

  5. Overview of MMPose Framework Components

    main

    MMPose is an open-source toolbox based on PyTorch for pose estimation. It consists of 8 main components:

    • apis: High-level APIs for model inference.
    • structures: Data structures including bbox, keypoint, and PoseDataSample.
    • datasets: Support for various pose estimation datasets, including transforms for data augmentation.
    • codecs: Pose encoders (e.g., converting keypoints to heatmaps) and decoders (e.g., converting model output back to pose estimates).
    • models: Modular components for pose estimation models, including:
      • pose_estimators: The main model classes.
      • data_preprocessors: Input data preprocessing.
      • backbones: Backbone networks.
      • necks: Model neck components.
      • heads: Model head components.
      • losses: Loss functions.
    • engine: Runtime components, including hooks for runtime execution.
    • evaluation: Metrics for evaluating model performance.
    • visualization: Tools for visualizing keypoint skeletons, heatmaps, and other information.

  6. Overview of YOLO-Pose (YOLOX-Pose)

    main

    YOLO-Pose is a bottom-up pose estimation approach that performs multi-person pose estimation by simultaneously detecting person instances and regressing keypoint locations in a single pass.

    In mmpose, we implement YOLOX-Pose, which is based on the YOLOX object detection framework. It utilizes separate branches with adaptive convolutions to regress offsets for different joints, allowing for optimized feature extraction for each specific keypoint.

  8. Overview of Top-down heatmap-based pose estimation

    main

    Top-down methods for fashion pose estimation follow a two-stage process:

    1. Object Detection: Identifying the object within the image.
    2. Single-object Pose Estimation: Estimating keypoints within the provided object bounding boxes.

    Instead of predicting raw keypoint coordinates, these models generate heatmaps that represent the likelihood of a keypoint being present at specific locations.

  9. Overview of Top-down Heatmap-based Animal Pose Estimation

    main

    Top-down heatmap-based pose estimation follows a two-stage process:

    1. Object Detection: Identifying the object within the image.
    2. Single-object Pose Estimation: Estimating keypoints within the provided object bounding boxes.

    Instead of predicting raw keypoint coordinates, this method produces heatmaps representing the likelihood of keypoint locations, following the paradigm established in 'Simple Baselines for Human Pose Estimation and Tracking'.

  10. Overview of RTMPose toolkit

    main

    RTMPose is a high-performance, real-time multi-person pose estimation toolkit built on top of MMPose. It is designed for industrial applications where low latency and high accuracy are critical. The toolkit provides models of varying scales (t, s, m, l) to balance performance and speed across different hardware environments.

    Key Features:

    • High Accuracy & Low Latency: Models achieve high COCO AP while maintaining high FPS on both CPU and GPU.
    • Easy Deployment: Supports multiple backends including ONNX, TensorRT, ncnn, and OpenVINO via MMDeploy.
    • Cross-Platform Support: Compatible with Linux, Windows, NVIDIA Jetson, ARM, and mobile platforms.
    • Business-Ready: Provides inference interfaces and SDKs for Python, C++, C#, and Java.

  12. Overview of MMPose components

    main

    MMPose is a PyTorch-based pose estimation toolkit for 2D multi-person human pose estimation, hand pose estimation, face landmark detection, whole-body human pose estimation, fashion landmark detection, and animal pose estimation. The framework is organized into several core components:

    • apis: High-level APIs for model inference.
    • structures: Data structures such as bbox, keypoint, and PoseDataSample.
    • datasets: Support for various pose estimation datasets, including transforms for data augmentation.
    • codecs: Pose encoders (e.g., encoding keypoints into heatmaps) and decoders (decoding model outputs into pose predictions).
    • models: Modular components including:
      • pose_estimators: Model classes.
      • data_preprocessors: Input data preprocessing.
      • backbones: Backbone networks.
      • necks: Neck modules.
      • heads: Prediction heads.
      • losses: Loss functions.
    • engine: Runtime components and hooks for the runner.
    • evaluation: Metrics for performance evaluation.
    • visualization: Tools for visualizing skeletons, heatmaps, and other information.