EasyMocap

When performing conversions, be aware of the differences between C++11 clock types, as they affect the resulting Python types:

• std::chrono::system_clock: Measures current date and time. It can change based on OS updates (e.g., NTP synchronization). Converting this to Python results in datetime.datetime objects.
• std::chrono::steady_clock: Ticks at a steady rate and is never adjusted. It is ideal for timing but does not correspond to the current date/time. Converting this to Python results in datetime.timedelta objects.
• std::chrono::high_resolution_clock: Provides the highest resolution available. It may be a typedef of system_clock (resulting in datetime.datetime in Python) or steady_clock (resulting in datetime.timedelta in Python), depending on the system.

The mv1p.py pipeline operates in two main phases:

Phase 1: Triangulation

Converts multi-view 2D keypoints to 3D keypoints using SVD/DLT.

• Input: Images, 2D annotations (Body25), and camera parameters (intri.yml, extri.yml).
• Output: keypoints3d/*.json.

Phase 2: SMPL Fitting

Optimizes SMPL parameters based on the 3D and 2D data.

1. optimizeShape: Estimates body shape from bone lengths.
2. optimizePose3D: Optimizes global rotation (Rh), translation (Th), and 3D poses from 3D keypoints.
3. optimizePose2D: Fine-tunes parameters using 2D keypoint constraints.

• Output: smpl/*.json and various visualizations.

Once chessboard corners are extracted and verified, run the intrinsic calibration. This step may take a significant amount of time.

python3 apps/calibration/calib_intri.py ${data} --step 5

Output: An intri.yml file will be generated in ${data}/output.

Use the extract_video.py script to preprocess your videos and integrate OpenPose for keypoint detection. You can specify the OpenPose path and enable hand/face detection.

data=path/to/data
out=path/to/output
python3 scripts/preprocess/extract_video.py ${data} --openpose <openpose_path> --handface

If a C++ std::string contains data using a different encoding (e.g., Latin-1), you must perform an explicit conversion to a py::str object to avoid UnicodeDecodeError in Python. You can use the Python C API (e.g., PyUnicode_DecodeLatin1) or a third-party library like libiconv to transcode to UTF-8.

// This uses the Python C API to convert Latin-1 to Unicode
m.def("str_output",
    []() {
        std::string s = "Send your r\xe9sum\xe9 to Alice in HR"; // Latin-1
        py::str py_s = PyUnicode_DecodeLatin1(s.data(), s.length());
        return py_s;
    }
);

If you need to generate pybind11 binding code automatically, you can use the Binder project. It introspects existing C++ codebases using LLVM/Clang to automate the process. Refer to the official Binder documentation for detailed usage instructions.

http://cppbinder.readthedocs.io/en/latest/about.html

If a C++ library uses std::cout or std::cerr, its output won't automatically appear in Python's sys.stdout or sys.stderr. You can use py::scoped_ostream_redirect or py::scoped_estream_redirect to redirect these streams. These guards respect flushes and allow real-time redirection (e.g., to a Jupyter notebook).

Note: These methods do not redirect C-level output like fprintf. For that, you must redirect file descriptors using os.dup2 or OS-specific C calls.

#include <pybind11/iostream.h>

// Add a scoped redirect for your noisy code
m.def("noisy_func", []() {
    py::scoped_ostream_redirect stream(
        std::cout,                               // std::ostream&
        py::module::import("sys").attr("stdout") // Python output
    );
    call_noisy_func();
});

Fit the SMPL body model to the tracked 3D keypoints. This uses the tracked keypoints generated by auto_track.py to produce the final SMPL model output.

python3 apps/demo/smpl_from_keypoints.py ${data} --skel ${data}/output-track/keypoints3d --out ${data}/output-track/smpl --verbose --opts smooth_poses 1e1

To use your own dataset, organize your files into the following directory structure:

<seq>
├── intri.yml
├── extri.yml
└── videos
    ├── 1.mp4
    ├── 2.mp4
    └── ...

• videos/: Contains the input video files.
• intri.yml: Stores camera intrinsic parameters.
• extri.yml: Stores camera extrinsic parameters.

For instructions on camera calibration and the specific format for camera parameters, refer to the apps/calibration documentation.