MyoSuite Documentation

MyoSuite provides a collection of musculoskeletal simulation tasks based on three core models:

• Finger: Used for joint pose and fingertip reaching tasks.
• Elbow: A simplified model (6 muscles) used for testing initial control schemes via elbow rotations.
• Hand: A full forearm-wrist-hand model used for complex, contact-rich tasks like key turning, object holding, pen twirling, and baoding balls.

Tasks range from simple reaching to highly dexterous movements requiring coordination and collision avoidance.

Several research projects and publications leverage the MyoSuite framework for studying reinforcement learning and dexterous manipulation:

• MyoDex: A generalizable prior for dexterous manipulation. Link
• DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems. Link
• Lattice: Latent Exploration for Reinforcement Learning. Link
• SAR: Generalization of Physiological Agility and Dexterity via Synergistic Action Representation. Link

If you have a project using MyoSuite, you can contribute by creating a Pull Request to include it in the documentation.

The Prosthesis Locomotion task involves a trans-femoral myoLeg model and an Open Source Leg (OSL) prosthetic. The goal is to learn the dynamics and control of a powered prosthetic leg to traverse different terrain types (Flat, Hilly, Rough, Stairs) with increasing difficulty.

Key Constraints:

• No Direct Control: You cannot directly observe or control the prosthetic leg's angles, velocities, or torques.
• Interaction: Interaction with the prosthetic leg occurs via socket interaction forces on the residual limb (provided in observations).
• Prosthetic Control: The prosthetic leg is controlled by an embedded state-machine-based impedance controller. Participants must update this controller's parameters to adapt to terrain.

The MyoChallenge provides two distinct environment types:

1. TEST environment: Open to participants for familiarization and development.
2. EVALUATION environment: The final environment used for scoring.

Note: While the tasks and evaluation criteria are identical, the distributions of changing factors may differ between TEST and EVALUATION. To ensure high performance, developers should focus on training policies that are robust across a wide range of distributions.

MyoSuite provides MJX and MJWarp implementations for accelerated training.

• MJX (MuJoCo XLA): Provides significant acceleration (up to 35x) for environments like MjxElbowPoseRandom-v0 and MjxFingerPoseRandom-v0. However, performance gains may be lower (2-5x) in contact-rich environments like MjxHandReachRandom-v0.
• MJWarp (MuJoCo Warp): Optimized for contact-rich environments. It consistently outperforms both MuJoCo and MJX, offering up to 100x speedup over MuJoCo and 5-20x speedup over MJX. Increasing the number of parallel environments consistently improves training time with Warp.

Simulation/rollout speeds for MuJoCo Warp can reach 150K to 1M steps per second depending on the environment and parallelism.

The Table Tennis Rally challenge requires an agent to hit a pingpong ball with a paddle so that it lands on the opponent's side. This task necessitates the coordination of a myoArm model and a myoTorso model to achieve accurate hits and sufficient force for a high-speed rally.

Objective: Develop a general policy to move the ball from the agent's side to the opposite side by hitting it with a paddle.

MyoSuite includes models for simulating physiological conditions and assistive technologies:

• Sarcopenia: Modeled as a 50% reduction of maximal isometric force for each muscle.
• Fatigue: A dynamic muscle fatigue model where a fatigue factor is applied to all muscles (simplified implementation).
• Tendon Transfer: Simulates surgery by redirecting the application point of muscle forces from one joint DoF to another (e.g., moving EIP application from index to thumb).
• Exoskeleton Assistance: Modeled for the elbow using an ideal actuator and two supports (upper arm: 0.101 Kg, forearm: 0.111 Kg). Assistance is provided as a percentage of biological joint torque.

IK-generated trajectories can be used in Myochallenge to improve training efficiency or provide expert demonstrations:

• Reference State Initialization: Collect trajectories that interpolate from distant poses to well-positioned states (e.g., a correctly positioned paddle). Starting episodes in these plausible poses makes control policy exploration more efficient.
• Imitation Learning: Use the extracted IK trajectories as ground truth for motion tracking or imitation learning algorithms.

To study robustness and physiological changes, MyoSuite environments support several non-stationarity variations. These can be applied to various tasks to simulate different physiological conditions:

• Sarcopenia: Simulated muscle loss/weakness.
• Fatigue: Simulated muscle fatigue.
• Tendon-transfer: Simulated surgical tendon transfers.

Most tasks (Finger, Elbow, and Hand) support Sarcopenia and Fatigue, while the more complex Hand tasks also support Tendon-transfer variations.