RoboVerse Documentation

RoboVerse provides standardized benchmarks for both imitation learning and reinforcement learning. The platform defines a unified training and evaluation protocol and provides standardized baselines and learning frameworks.

For imitation learning, RoboVerse specifically focuses on assessing model generalization through different levels of generalization benchmarks.

The following RL algorithms are available for training in RoboVerse environments:

• PPO (ppo.py): Proximal Policy Optimization.
• SAC (sac.py): Soft Actor-Critic.
• TD3 (td3.py): Twin Delayed Deep Deterministic Policy Gradient.

The migration of the Calvin task to RoboVerse is partially complete.

Completed features:

• Loading data from the Calvin dataset.
• Static environment setup for calvin_scene_A.
• Joint-state based control of the robot.

Pending features:

• Implementation of light and button functions in the Calvin Env.
• Splitting the large dataset into small clips saved as trajectories.
• Writing configuration files for all scenes (A, B, C, D).

Open6Dor is a benchmark for open-instruction 6-DoF object rearrangement. It consists of 5k+ tasks across position-track, rotation-track, and 6-DoF-track.

Assets: Objects are filtered and scaled from YCB and Objaverse-XL into a standardized mesh representation. Success Criteria: Success is determined by comparing the target object's final pose against the annotated ground-truth pose range.

For Flow Matching (FM), the repository includes several 1-NFE (1 number of function evaluations) generation methods:

• MeanFlow
• Improved MeanFlow (iMF)
• Consistency Flow Matching (CFM)

When setting up your data collection and training, you can switch between control modes by adjusting the action_space and observation_space parameters:

• Joint Position Control: Set both action_space and observation_space to joint_pos.
• End Effector Control: Set both action_space and observation_space to ee.
  • Use delta_ee=1 for delta control.
  • Use delta_ee=0 for absolute positioning.

Note on state_dim: While the original ACT paper uses an action joint space of 14, this implementation allows a parameterized state_dim. It defaults to 9, which is compatible with both Franka joint space and end effector space.

RoboVerse supports multiple VLA architectures depending on your resource constraints and accuracy requirements:

• OpenVLA: Full-size model using RLDS. Detailed in OpenVLA/ directory.
• SmolVLA: Lightweight model from Hugging Face/LeRobot. Detailed in SmolVLA/README.md.
• π0 family: Physical Intelligence's models. Detailed in pi0/README.md.

The framework supports customizable terrain generation via metasim.scenario.grounds. Predefined configurations are in roboverse_pack/grounds/.

Supported Terrain Types:

• SlopeCfg: Planar inclined surfaces.
• StairCfg: Staircase features.
• ObstacleCfg: Random rectangular obstacle fields.
• StoneCfg: Stone-like protrusions.
• GapCfg: Gaps to traverse.
• PitCfg: Rectangular pits.

Example Configuration:

from metasim.scenario.grounds import GroundCfg, SlopeCfg, StairCfg

ground_cfg = GroundCfg(
    width=20.0,
    length=20.0,
    horizontal_scale=0.1,
    vertical_scale=0.005,
    static_friction=1.0,
    dynamic_friction=1.0,
    elements={
        "slope": [SlopeCfg(origin=[0, 0], size=[2.0, 2.0], slope=0.3)],
        "stair": [StairCfg(origin=[5, 0], size=[2.0, 2.0], step_height=0.1)]
    }
)