RoboCasa GR-1 Tabletop Tasks

Evaluation is performed by running an inference server in the Isaac-GR00T repository and then running a simulation service client.

1. Start the Inference Server

Inside the Isaac-GR00T directory, run:

cd Isaac-GR00T
python3 scripts/inference_service.py --server \
    --model_path <MODEL_PATH> \
    --data_config fourier_gr1_arms_waist

2. Run the Simulation Evaluation

In a separate terminal, inside the Isaac-GR00T directory, run:

cd Isaac-GR00T
python3 scripts/simulation_service.py --client \
    --env_name <TASK_NAME> \
    --video_dir ./videos \
    --max_episode_steps 720 \
    --n_envs 5 \
    --n_episodes 10

This will run the model for the specified number of episodes, collect success metrics, and save rollout videos to the --video_dir.

Available Task Names

Tasks follow the pattern gr1_unified/<TASK_NAME>_Env. Examples include:

• gr1_unified/PnPCupToDrawerClose_GR1ArmsAndWaistFourierHands_Env
• gr1_unified/PosttrainPnPNovelFromCuttingboardToBasketSplitA_GR1ArmsAndWaistFourierHands_Env
• gr1_unified/PosttrainPnPNovelFromPlateToPlateSplitA_GR1ArmsAndWaistFourierHands_Env

python3 scripts/simulation_service.py --client --env_name <TASK_NAME> --video_dir ./videos --max_episode_steps 720 --n_envs 5 --n_episodes 10

To use this repository, you must set up a Conda environment and install three core dependencies: Isaac-GR00T, robosuite, and robocasa-gr1-tabletop-tasks. Finally, you must download the required simulation assets.

Follow these steps in order:

1. Set up Conda environment

   conda create -c conda-forge -n robocasa python=3.10
   conda activate robocasa

2. Install Isaac-GR00T

   git clone https://github.com/NVIDIA/Isaac-GR00T.git
   cd Isaac-GR00T
   pip install --upgrade setuptools
   pip install -e .[base]
   pip install --no-build-isolation flash-attn==2.7.1.post4 
   cd ..

3. Install robosuite

   git clone https://github.com/ARISE-Initiative/robosuite.git
   pip install -e robosuite

4. Install robocasa-gr1-tabletop-tasks

   git clone https://github.com/robocasa/robocasa-gr1-tabletop-tasks.git
   pip install -e robocasa-gr1-tabletop-tasks

5. Download assets

   cd robocasa-gr1-tabletop-tasks
   python robocasa/scripts/download_tabletop_assets.py -y

# 1. Set up conda environment
conda create -c conda-forge -n robocasa python=3.10
conda activate robocasa

# 2. Clone and install Isaac-GR00T
git clone https://github.com/NVIDIA/Isaac-GR00T.git
cd Isaac-GR00T
pip install --upgrade setuptools
pip install -e .[base]
pip install --no-build-isolation flash-attn==2.7.1.post4 
cd ..

# 3. Clone and install robosuite
git clone https://github.com/ARISE-Initiative/robosuite.git
pip install -e robosuite

# 4. Clone and install robocasa-gr1-tabletop-tasks
git clone https://github.com/robocasa/robocasa-gr1-tabletop-tasks.git
pip install -e robocasa-gr1-tabletop-tasks

# 5. Download assets
cd robocasa-gr1-tabletop-tasks
python robocasa/scripts/download_tabletop_assets.py -y

When adding a new robot to Robocasa, the MJCF file must be placed in robocasa/models/assets/robots/ and adhere to specific structural rules to ensure compatibility with the framework's kinematic and end-effector logic:

• Root Element: The MJCF must have a single root element containing all other elements (e.g., <worldbody>, <asset>).
• Kinematic Tree: The tree must start from a <body name="base"> element. This body must contain a freejoint and be placed directly under <worldbody>.
• Joint Ordering: Joint definitions for the right arm must appear in the XML structure before the joint definitions for the left arm.
• End Effectors: Gripper attachments are only supported if they are attached to a body named <body name="right_eef">.
• Fixed-Base Mode: If operating in fixed-base mode, the body connected to the base joint will be anchored to the world frame. Select a mounting point (typically the pelvis or torso) that ensures correct fixed-base behavior.

You can fine-tune the GR00T-N1.5-3B model using the gr00t_finetune.py script. This is typically done on high-performance hardware (e.g., H100 Node with 8 GPUs).

#!/bin/bash
ALL_DATASET_PATHS=(
  "YOUR_DATASET_PATH_1"
  "YOUR_DATASET_PATH_2"
)

python scripts/gr00t_finetune.py \
  --dataset-path "${ALL_DATASET_PATHS[@]}" \
  --num-gpus 8 --batch-size 60 --learning_rate 3e-5 \
  --output-dir OUTPUT_DIR \
  --data-config fourier_gr1_arms_waist --embodiment_tag gr1 \
  --tune-visual \
  --max-steps 30000 --save-steps 5000

Arguments:

• --dataset-path: Space-separated list of paths to datasets.
• --num-gpus: Number of GPUs to use.
• --batch-size: Training batch size.
• --learning_rate: Learning rate.
• --output-dir: Directory to save training results.
• --data-config: Configuration key for the data format (e.g., fourier_gr1_arms_waist).
• --embodiment_tag: Tag identifying the robot embodiment (e.g., gr1).
• --tune-visual: (Optional) Flag to tune the visual encoder.
• --max-steps: Total training steps.
• --save-steps: Frequency of checkpoint saving.