DOCSEARCHbyAlgolia
Submit Docs

Isaac Gym Benchmark Environments

repository·main·Indexed 25 days ago

https://github.com/isaac-sim/isaacgymenvs

A collection of high-performance, GPU-accelerated physics simulation environments for robot learning and reinforcement learning research. It includes the poselib library for skeleton pose manipulation and retargeting, tools for creating vectorized environments via isaacgymenvs.make, and support for training RL policies using Hydra configurations. The repository also features DeXtreme environments for sim-to-real transfer using Manual and Automatic Domain Randomization (ADR).

Tokens
12.5K
Snippets
25
Records
73
Agent score
84%

What's inside isaacgymenvs

  1. Overview of the poselib library

    main

    poselib is a library for loading, manipulating, and retargeting skeleton poses and motions. It is built on top of PyTorch and requires data to be in PyTorch tensors. The library is organized into three modules:

    • poselib.core: Basic data loading and tensor operations.
    • poselib.skeleton: Higher-level skeleton operations (topology, states, and motions).
    • poselib.visualization: Tools for displaying skeleton poses.

  2. Implement DeXtreme with Manual or Automatic Domain Randomization

    main

    DeXtreme provides environments for transferring cube rotation tasks with an Allegro hand from simulation to the real world. It offers two variants for sim-to-real training:

    1. Manual Domain Randomization (ManualDR): Uses user-defined parameter ranges for randomization.
    2. Automatic Domain Randomization (ADR): Automatically updates parameter ranges based on periodic simulation performance benchmarking.

    Both variants use Asymmetric Actor-Critic training, where the policy receives only real-world available inputs, while the value function receives privileged simulator information. These environments use dictionary observations (enabled via use_dict_obs=True) to manage inputs cleanly.

    Key classes are located in tasks/dextreme/allegro_hand_dextreme.py and tasks/dextreme/adr_vec_task.py.

  3. Implement custom RL policies using Factory templates

    main

    For tasks that do not train RL policies by default (FactoryTaskNutBoltInsertion and FactoryTaskNutBoltGears), use the provided scripts as templates for your own RL implementation.

    To see a completed example of how to fill out a template, refer to the FactoryTaskNutBoltPick script.

  4. Train RL policies using `train.py`

    main

    Run the train.py script to start training a policy for a specific task. By default, a preview window is shown. You can use the v key to toggle the viewer to speed up training, or esc to stop.

    Basic usage:

    python train.py task=Cartpole

    Headless mode (faster, no viewer):

    python train.py task=Ant headless=True
    python train.py task=Ant

  5. Train DeXtreme with Manual Domain Randomization

    main

    To train DeXtreme RL policies using Manual DR on a single GPU, use the following command. This example includes specific overrides for success tolerance, network architecture (LSTM), and randomized quaternion goals.

    HYDRA_MANUAL_DR="train.py multi_gpu=False \
task=AllegroHandDextremeManualDR \
task.env.resetTime=8 task.env.successTolerance=0.4 \
experiment='allegrohand_dextreme_manual_dr' \
headless=True seed=-1 \
task.env.startObjectPoseDY=-0.15 \
task.env.actionDeltaPenaltyScale=-0.2 \
task.env.resetTime=8 \
task.env.controlFrequencyInv=2 \
train.params.network.mlp.units=[512,512] \
train.params.network.rnn.units=768 \
train.params.network.rnn.name=lstm \
train.params.config.central_value_config.network.mlp.units=[1024,512,256] \
train.params.config.max_epochs=50000 \
task.env.apply_random_quat=True"

python ${HYDRA_MANUAL_DR}

  6. Implement Domain Randomization in a Task Class

    main

    To apply the YAML-defined randomizations to your simulation, follow these three steps in your task class (which should inherit from VecTask):

    1. Initialize Parameters: In your __init__ method, store the randomization parameters from the config: self.randomization_params = self.cfg["task"]["randomization_params"]

    2. Initial Pass (Setup): Call self.apply_randomizations once inside your create_sim() method. This is necessary for properties like mass or scale that require setup_only: True.

    3. Update Schedule: In post_physics_step(), increment the randomize_buf tensor to support scheduled randomizations: self.randomize_buf += 1

    4. Apply at Reset: Call self.apply_randomizations during the environment reset phase: self.apply_randomizations(self.randomization_params)

  7. Launch Reinforcement Learning training

    main

    Single-GPU training for reinforcement learning examples can be launched using python train.py.

    When training with the viewer enabled (non-headless mode), you can press v to toggle viewer sync. Disabling viewer sync can improve performance, particularly in GPU pipeline mode, while re-enabling it allows you to check training progress.

    python train.py

  8. Load and test trained checkpoints

    main

    Checkpoints are stored in runs/EXPERIMENT_NAME/nn. You can load them for continued training or for inference only.

    Continue training from a checkpoint:

    python train.py task=Ant checkpoint=runs/Ant/nn/Ant.pth

    Perform inference only (testing): Set test=True. You may also want to reduce num_envs to minimize rendering overhead.

    python train.py task=Ant checkpoint=runs/Ant/nn/Ant.pth test=True num_envs=64

    Note: If checkpoint paths contain special characters like [ or =, wrap the path in quotes and escape the characters (e.g., checkpoint="./runs/Ant/nn/last_Antep\=501rew\[5981.31\].pth").

  9. Create a new RL task

    main

    To create a new task, follow these steps:

    1. Create a new script in isaacgymenvs/tasks/.
    2. Import necessary modules:
      from isaacgym import gymtorch, gymapi
from .base.vec_task import VecTask
    3. Define your class inheriting from VecTask:
      class MyNewTask(VecTask):
    def __init__(self, cfg, sim_device, headless):
        super().__init__(cfg=cfg)
        # Initialize state tensors here
        dof_state_tensor = self.gym.acquire_dof_state_tensor(self.sim)
        self.dof_state = gymtorch.wrap_tensor(dof_state_tensor)

    def create_sim(self): 
        # Implement sim setup, axis, ground plane, and env creation
        pass

    def pre_physics_step(self, actions):
        # Implement pre-physics logic (e.g., applying actions)
        pass

    def post_physics_step(self): 
        # Implement post-physics logic (e.g., computing rewards/obs)
        pass
    4. Register the task in isaacgymenvs/tasks/__init__.py by adding it to the isaac_gym_task_map dictionary.
    5. Create configuration files:
      • Task Config: Create [TaskName].yaml in isaacgymenvs/cfg/task/. The name in the root must match the task name in the map.
      • Train Config: Create [TaskName]PPO.yaml in isaacgymenvs/cfg/train/ (using the PPO suffix for rl_games).
    6. Run the task:
      python train.py task=MyNewTask
    class MyNewTask(VecTask):
    def __init__(self, cfg, sim_device, headless):
        super().__init__(cfg=cfg)
        dof_state_tensor = self.gym.acquire_dof_state_tensor(self.sim)
        self.dof_state = gymtorch.wrap_tensor(dof_state_tensor)

    def create_sim(self):
        pass

    def pre_physics_step(self, actions):
        pass

    def post_physics_step(self):
        pass

  10. Importing MJCF assets to SkeletonTree

    main

    You can import MJCF (robotics file format) assets into SkeletonTree definitions to represent skeleton topology. This is useful for retargeting motion sequences to simulation skeletons created in MJCF format. Importing to SkeletonTree allows you to generate T-poses or other retargeting poses.

    See mjcf_importer.py for an example script.

  11. Run DeXtreme training on Multi-GPU

    main

    To scale training to multiple GPUs (e.g., a single DGX node), use torchrun. You must define the ${GPUS} environment variable (e.g., GPUS=8) before running the command.

    Manual DR Multi-GPU

    Set the multi_gpu=True flag and use the following command structure:

    torchrun --nnodes=1 --nproc_per_node=${GPUS} --master_addr '127.0.0.1' ${HYDRA_MANUAL_DR}

    ADR Multi-GPU

    Use the following command structure for ADR:

    torchrun --nnodes=1 --nproc_per_node=${GPUS} --master_addr '127.0.0.1' ${HYDRA_ADR}

  12. Enable PyTorch deterministic training

    main

    When running Reinforcement Learning (RL) training, you can use the torch_deterministic argument to force PyTorch to use deterministic algorithms.

    Warning: Enabling this may negatively impact runtime performance.

    Special Behavior: If both torch_deterministic=True and seed=-1 are set, the seed value will be automatically fixed to 42 instead of being random.

    Compatibility Note: In PyTorch versions 1.9 and 1.9.1, enabling torch_deterministic may cause crashes due to bugs in those specific PyTorch versions.