DOCSEARCHbyAlgolia
Submit Docs

Momentum Human Rig (MHR)

repository·main·Indexed 20 days ago

https://github.com/facebookresearch/mhr

A high-fidelity, parametric 3D human body model providing identity, pose, and facial expression parameterization. Designed for real-time applications with GPU acceleration and compatibility with the PyMomentum solver, MHR supports multiple Levels of Detail (LOD 0-6) and provides tools for mesh generation, LOD conversion, and visualization via Jupyter notebooks.

Tokens
7K
Snippets
22
Records
30
Agent score
73%

What's inside MHR

  1. Compare PyMomentum and PyTorch optimization methods

    main

    The conversion tool supports two optimization backends, each suited for different use cases:

    PyMomentum

    Best for: Scenarios requiring temporal consistency and robust fitting.

    • Pros: Robust hierarchical optimization; supports temporal tracking via is_tracking=True.
    • Cons: CPU-only; may be slower for large batches of temporally inconsistent data. The identity is the average identity across the first sequential processing, not optimized across the whole sequence.

    PyTorch

    Best for: Large-scale conversion of independent poses.

    • Pros: GPU-accelerated; faster processing.
    • Cons: Processes each frame independently; does not leverage temporal consistency.

  2. Install MHR using Pixi (Recommended)

    main

    The recommended way to install MHR is using Pixi, a conda-based package manager. This ensures all dependencies, including pymomentum, are correctly resolved. Follow these steps:

    1. Clone the repository.
    2. Install dependencies using pixi install.
    3. Download and unzip model assets using pixi run download-assets.
    4. Activate the environment with pixi shell.
    # Clone the repository
git clone git@github.com:facebookresearch/MHR.git
cd MHR

# Install dependencies with Pixi
pixi install

# Download and unzip the model assets
pixi run download-assets

# Activate the environment
pixi shell

  5. Install MHR using pip (Experimental)

    main

    Installing via pip is considered experimental. Some dependencies like pymomentum may not resolve correctly on all platforms. If you encounter issues, use the Pixi installation method instead.

    1. Install either pymomentum-cpu or pymomentum-gpu.
    2. Install the mhr package.
    3. Download assets using mhr-download-assets.
    # Install PyMomentum (CPU or GPU)
pip install pymomentum-cpu  # or pymomentum-gpu

# Install MHR
pip install mhr

# Download and unzip the model assets
mhr-download-assets

  6. Install MHR using the TorchScript model

    main

    If you want a lightweight installation without requiring the full codebase or local model assets, you can use the TorchScript model.

    Note: This method currently only supports LOD 1 and provides limited access to model properties.

    1. Install the mhr package via pip.
    2. Download the TorchScript model asset using the mhr-download-assets CLI.
    3. Use the model as a standalone PyTorch Graph mode model.
    # Install MHR
pip install mhr

# Download the torchscript model
mhr-download-assets --member assets/mhr_model.pt --output mhr_model.pt

  7. Use the MHR Visualization Notebook in Google Colab

    main

    You can run the MHR visualization notebook in Google Colab using one of the following three methods:

    Option 1: Direct URL (Recommended)

    Open the notebook directly in Colab by clicking this link: https://colab.research.google.com/github/facebookresearch/MHR/blob/main/tools/mhr_visualization/MHR%20visualization.ipynb

    Option 2: Upload to Google Colab

    1. Go to Google Colab.
    2. Select File → Upload notebook.
    3. Select the Upload tab.
    4. Click Choose File and select MHR visualization.ipynb from your local machine.

    Option 3: Open from Google Drive

    1. Upload MHR visualization.ipynb to your Google Drive.
    2. Right-click the file in Google Drive.
    3. Select Open with → Google Colaboratory. Note: If Colaboratory is not listed, click Connect more apps, search for "Colaboratory", and install it.

    Once the notebook is open in Colab, follow the internal instructions within the notebook cells to execute the visualization code.

  8. Install the SMPL-MHR Conversion Tool

    main

    To use the conversion tool, you must first have the MHR environment set up. Install the required Python dependencies using pixi:

    pixi add --pypi trimesh scikit-learn tqdm smplx

    You also need the official SMPL and SMPLX model files, which must be downloaded separately from their respective websites:

    Tip: If you encounter issues with .pkl model files, use the official .npz files instead.

    pixi add --pypi trimesh scikit-learn tqdm smplx

  9. Convert MHR mesh vertices between LOD levels

    main

    Use the LODConverter class to transform mesh vertices from one Level of Detail (LOD) to another using precomputed barycentric mappings.

    To use the converter:

    1. Initialize LODConverter with the source_lod level.
    2. Load your source vertices as a NumPy array with shape (num_verts, 3).
    3. Call .convert() specifying the target_lod.
    4. The resulting mesh object can be saved using .export().
    import numpy as np
from example import LODConverter

converter = LODConverter(source_lod=1)

# Provide your own source vertices (shape: (num_verts, 3))
src_verts = np.load("my_vertices.npy")
mesh = converter.convert(src_verts, target_lod=6)
mesh.export("converted_lod6.ply")

  10. How the MHRPoseCorrectivesModel works

    main

    The MHRPoseCorrectivesModel is a non-linear neural network module that predicts pose-dependent vertex offsets (correctives) to improve mesh quality during articulation.

    It operates by:

    1. Taking joint_parameters (local per-joint transforms) as input.
    2. Extracting pose features using _pose_features_from_joint_params, which converts Euler rotations into a 6D representation (via batch6DFromXYZ).
    3. Passing these features through a pose_dirs_predictor (a torch.nn.Sequential network).
    4. Reshaping the output into pose_corrective_offsets of shape [batch, num_joints, 3].

    These offsets are added to the unposed mesh (rest pose) before skinning is applied in the main MHR model.

    class MHRPoseCorrectivesModel(torch.nn.Module):
    # Predicts non-linear pose correctives offsets
    def forward(self, joint_parameters: torch.Tensor) -> torch.Tensor:
        # ... returns pose_corrective_offsets
        pass

  11. Use the MHR model for mesh generation

    main

    You can use the MHR class to generate 3D human meshes and skeleton information by providing identity, pose, and facial expression parameters. The model supports multiple Levels of Detail (LOD 0-6).

    import torch
from mhr.mhr import MHR

# Load MHR model (LOD 1, on CPU)
mhr_model = MHR.from_files(device=torch.device("cpu"), lod=1)

# Define parameters
batch_size = 2
identity_coeffs = 0.8 * torch.randn(batch_size, 45)      # Identity
model_parameters = 0.2 * (torch.rand(batch_size, 204) - 0.5)  # Pose
face_expr_coeffs = 0.3 * torch.randn(batch_size, 72)     # Facial expression

# Generate mesh vertices and skeleton information (joint orientation and positions).
vertices, skeleton_state = mhr_model(identity_coeffs, model_parameters, face_expr_coeffs)

  12. Programmatic usage of the Conversion API

    main

    To integrate the conversion tool into your Python code, use the Conversion class. You must initialize both an MHR model and an smplx model before creating the converter.

    import torch
from mhr.mhr import MHR
from smpl_mhr import Conversion
import smplx

# Initialize models
mhr_model = MHR.from_files(lod=1, device="cuda")
smplx_model = smplx.SMPLX(model_path="path/to/smplx", gender="neutral")

# Create converter
converter = Conversion(
    mhr_model=mhr_model,
    smpl_model=smplx_model,
    method="pytorch"  # or "pymomentum"
)

# Example: Convert SMPLX to MHR
results = converter.convert_smpl2mhr(
    smpl_parameters=smplx_params,
    single_identity=True,
    return_mhr_meshes=True,
    return_mhr_parameters=True
)

# Example: Convert MHR back to SMPLX
smplx_results = converter.convert_mhr2smpl(
    mhr_parameters=results.result_parameters,
    return_smpl_meshes=True
)
    import torch
from mhr.mhr import MHR
from smpl_mhr import Conversion
import smplx

mhr_model = MHR.from_files(lod=1, device="cuda")
smplx_model = smplx.SMPLX(model_path="path/to/smplx", gender="neutral")

converter = Conversion(
    mhr_model=mhr_model,
    smpl_model=smplx_model,
    method="pytorch"
)

results = converter.convert_smpl2mhr(
    smpl_parameters=smplx_params,
    single_identity=True,
    return_mhr_meshes=True,
    return_mhr_parameters=True
)

smplx_results = converter.convert_mhr2smpl(
    mhr_parameters=results.result_parameters,
    return_smpl_meshes=True
)