manotorch

Standard MANO uses a kinematic tree with 16 joints using axis-angle rotations. manotorch introduces an Anatomical Consistent Basis to represent joint rotations in a 3D Euclidean space using three orthogonal axes (twist-spread-bend) that satisfy hand anatomy.

Anatomy Loss

To prevent abnormal poses, manotorch provides AnatomyConstraintLossEE (Euler Angle version).

Key Improvements in manotorch (>= v0.0.2):

1. Stable Basis Calculation: Instead of calculating axes from a potentially already-abnormal posed hand, manotorch calculates the twist-spread-bend axes on a flat hand in its canonical pose. These are then transformed to the posed hand using the 16 $\mathbf{SE}(3)$ transformation matrices.
2. Robustness to Small Angles: It penalizes rotations using Euler angles, which is more robust than axis-angle when the rotation magnitude is near zero.

To see this in action, run:

python scripts/simple_anatomy_loss.py

To install manotorch, follow these steps:

1. Clone the repository and install dependencies:

   git clone https://github.com/lixiny/manotorch.git
   cd manotorch
   # For a new environment:
   conda env create -f environment.yaml
   # For an existing environment:
   conda env update -f environment.yaml

2. Download MANO pickle data-structures:

   • Visit the MANO website and create an account.
   • Download the Models and Code (file format mano_v*_*.zip).
   • Unzip the contents and copy the files from the mano_v*_*/ folder into the assets/mano/ directory of the manotorch folder.
   • Ensure your assets/mano structure includes models/ containing MANO_LEFT.pkl, MANO_RIGHT.pkl, SMPLH_female.pkl, and SMPLH_male.pkl.

3. (Optional) Install as a package: To use manotorch in other projects via imports, run the following from the manotorch root directory:

   pip install .

$ git clone https://github.com/lixiny/manotorch.git
$ cd manotorch
$ conda env create -f environment.yaml
$ pip install .

The joint order and tip indices in Omid's MANO implementation differ from manotorch and Yana's implementation. To convert Omid's output to match the manotorch format, you must:

1. Remove the tips from Omid's joint definition.
2. Extract the specific tip vertices using the manotorch tip indices.
3. Concatenate the original joints with the new tips.
4. Reorder the resulting joint array using the REORDER_IDX mapping.

manotorch Tip Indices:

• thumb: 745
• index: 317
• middle: 444
• ring: 556
• pinky: 673

Reorder Mapping (REORDER_IDX): [0, 13, 14, 15, 16, 1, 2, 3, 17, 4, 5, 6, 18, 10, 11, 12, 19, 7, 8, 9, 20]

# Assuming V3 and J3 are outputs from Omid's MANO model
REORDER_IDX = [0, 13, 14, 15, 16, 1, 2, 3, 17, 4, 5, 6, 18, 10, 11, 12, 19, 7, 8, 9, 20]

# 1. Remove the tips in Omid's definition (first 16 joints)
J3_ = J3[:, :16] 

# 2. Get the tips from manotorch/Yana's definition using vertex indices
tips = V3[:, [745, 317, 444, 556, 673]] 

# 3. Concatenate
J3_ = torch.cat([J3_, tips], 1)

# 4. Reorder to match manotorch
new_J3 = J3_[:, REORDER_IDX]

The ManoLayer is a differentiable PyTorch layer that maps pose and shape parameters to hand joints and vertices.

Initialization

from manotorch.manolayer import ManoLayer

# ncomps is the number of principal components for pose space
ncomps = 15
mano_layer = ManoLayer(use_pca=True, flat_hand_mean=False, ncomps=ncomps)

Forward Pass

Pass random_pose (shape [B, 3 + ncomps]) and random_shape (shape [B, 10]) to the layer.

Output Structure

The layer returns a MANOOutput namedtuple containing:

• verts: (B, 778, 3) vertices in meters, root relative.
• joints: (B, 21, 3) joints in meters, root relative.
• center_idx: Index of the center joint.
• center_joint: The center joint itself.
• full_poses: Full pose parameters.
• betas: Shape parameters.
• transforms_abs: (B, 16, 4, 4) SE3 transforms of each articulation, root relative.

import torch
from manotorch.manolayer import ManoLayer, MANOOutput

ncomps = 15
mano_layer = ManoLayer(use_pca=True, flat_hand_mean=False, ncomps=ncomps)

batch_size = 2
random_shape = torch.rand(batch_size, 10)
random_pose = torch.rand(batch_size, 3 + ncomps)

# forward mano layer
mano_output: MANOOutput = mano_layer(random_pose, random_shape)

verts = mano_output.verts
joints = mano_output.joints
transforms_abs = mano_output.transforms_abs

The manotorch.manolayer.ManoLayer can be initialized to use PCA (Principal Component Analysis) for pose representation. When using PCA, the random_pose input should include 3 values for global axis-angle rotation followed by the PCA components.

Key Parameters:

• rot_mode: Set to "axisang" for axis-angle rotation.
• use_pca: Set to True to enable PCA mode.
• ncomps: The number of principal components.
• mano_assets_root: Path to the MANO assets.
• side: Specifies the hand side (e.g., "right").
• flat_hand_mean: Boolean to enable flat hand mean.

Note on Units: Unlike some implementations that use millimeters, manotorch outputs vertices and joints in meters. If comparing against implementations that use millimeters (like manopth), you must divide the results by 1000.0.

import manotorch.manolayer as ThisMano

# Configuration
ncomps = 15
batch_size = 10

# Prepare inputs
# random_pose includes 3 global rotation values + ncomps
random_pose = torch.rand(batch_size, 3 + ncomps)
random_shape = torch.rand(batch_size, 10)

# Initialize layer
this_mano_layer = ThisMano.ManoLayer(
    rot_mode="axisang",
    center_idx=None,
    side="right",
    mano_assets_root="../assets/mano",
    use_pca=True,
    flat_hand_mean=True,
    ncomps=ncomps
)

# Forward pass
out = this_mano_layer(random_pose, random_shape)
# Access results via .verts and .joints
V2, J2 = out.verts, out.joints