DOCSEARCHbyAlgolia
Submit Docs

Project Instinct InstinctLab

repository·main·Indexed 20 days ago

https://github.com/project-instinct/instinctlab

An environment-side repository and template for Isaac Lab/Omniverse designed to industrialize Reinforcement Learning for humanoid whole-body control. It supports various tasks including Parkour, BeyondMimic, and Perceptive Shadowing, specifically providing robot descriptions and training configurations for the Unitree G1 robot.

Tokens
18.3K
Snippets
56
Records
82
Agent score
71%

What's inside InstinctLab

  1. Available Unitree G1 robot descriptions

    main

    The unitree_g1 package provides streamlined robot descriptions in URDF and MJCF formats for the Unitree G1 robot. There are four distinct versions available depending on your simulation requirements:

    • g1_23dof: 23 Degrees of Freedom version.
    • g1_29dof: 29 Degrees of Freedom version.
    • g1_29dof_with_hand: 29 Degrees of Freedom version including hands.
    • g1_dual_arm: Dual arm configuration.

  2. Configure a Multi Reward Manager for multi-critic RL

    main

    The MultiRewardManager replaces the default reward manager to support advantage-mixing or multi-critic Reinforcement Learning. It allows you to define multiple reward groups, where each group is a configuration class containing RewardTermCfg instances.

    Key behaviors:

    • The manager computes rewards separately for each group.
    • It is automatically used if the rewards configuration is an instance of MultiRewardCfg.
    • It returns a dictionary with shape (num_envs, num_groups).
    • Each group supports a combine_method: "sum" (default) or "prod" to combine individual terms.
    from instinctlab.managers import MultiRewardCfg
from isaaclab.managers import RewardTermCfg as RewTermCfg
from isaaclab.utils import configclass
import instinctlab.envs.mdp as instinct_mdp

@configclass
class RewardsCfg:
    # Individual reward terms
    base_position_imitation_gauss = RewTermCfg(
        func=instinct_mdp.base_position_imitation_gauss,
        weight=0.5,
        params={"std": 0.3},
    )

@configclass
class RewardGroupsCfg(MultiRewardCfg):
    # Define multiple groups for multi-critic RL
    rewards_group_1 = RewardsCfg()  # First critic
    rewards_group_2 = RewardsCfg()    # Second critic

@configclass
class EnvCfg(InstinctLabRLEnvCfg):
    rewards: RewardGroupsCfg = RewardGroupsCfg()

  3. How NoisyGroupedRayCasterCamera works

    main

    The NoisyGroupedRayCasterCamera extends GroupedRayCasterCamera to facilitate sim-to-real transfer by adding a configurable noise pipeline and temporal history buffers.

    Core features:

    • Noise Pipeline: Applies transformations to sensor data (like depth images). Supported noise types include Depth Artifacts, Depth Stereo Noise, Depth Sky Artifacts, Latency Noise, Gaussian/Uniform Noise, and Normalization.
    • History Buffers: Maintains temporal history for configured data types, enabling latency simulation (sampling from past frames), temporal filtering, and motion estimation.
    • Dual Output: Provides both clean (data_type) and noisy (data_type_noised) outputs for comparison or training.
    • Selective Noise: Noise can be applied selectively to specific sensor outputs (e.g., distance_to_image_plane, normals) via configuration.

  4. Project Instinct Contributor License Agreement terms

    main

    By submitting a pull request to Project Instinct, you agree to the following legal terms:

    • Copyright Transfer: You grant maintainers a perpetual, worldwide, non-exclusive, royalty-free, irrevocable copyright license to use, reproduce, modify, distribute, and create derivative works of your contributions.
    • License: All contributions are licensed under the CC BY-NC 4.0 license.
    • Attribution: Contributors are acknowledged in the project's contributor list, but copyright ownership transfers to the project maintainers.
    • No Commercial Use: Due to the CC BY-NC 4.0 license, all contributions are subject to a prohibition on commercial use.

  5. How Motion Reference data flows through the system

    main

    The motion reference system follows a specific data flow to enable imitation and tracking:

    1. Motion Buffers (files or generated) are loaded by the Motion Reference Manager.
    2. The Motion Reference Manager provides:
      • Motion Reference Data to the Shadowing Command.
      • Motion Reference Data with timing to *_tracking_* rewards.
      • Reference Frame (current timestep) to *_imitation_* rewards.
    3. The Shadowing Command provides:
      • Shadowing Command/Mask to Observations.
      • Shadowing Command/Mask to track_*_shadowing_cmd_* rewards.

  6. How GroupedRayCaster works

    main

    The GroupedRayCaster is an extension of the base RayCaster designed for dynamic environments. It allows ray casting against multiple meshes that move and update their positions during simulation.

    Key capabilities include:

    • Dynamic Mesh Tracking: It tracks rigid body views for each mesh group and updates transforms before each ray cast, ensuring rays hit moving objects correctly.
    • Collision Groups: Meshes and rays use collision group IDs. A group ID of -1 represents global meshes (like terrain) hit by all rays. Other IDs can be used to restrict rays to specific environment IDs, enabling parallel simulation isolation.
    • Multiple Mesh Sources: You can configure multiple mesh_prim_paths to cast rays against different sets of objects (e.g., terrain vs. robot parts).
    • Mesh Merging: It can merge multiple meshes from Xform prims into a single warp mesh, which is useful for complex articulated structures. This is configured via aux_mesh_and_link_names.

  8. Understand the BeyondMimic reward structure

    main

    The BeyondMimic approach uses Gaussian-based rewards to achieve smooth tracking. The reward system is composed of the following components:

    • Imitation Rewards (Gaussian):
      • Base position imitation
      • Base rotation imitation
      • Link position imitation (relative world frame)
      • Link rotation imitation (relative world frame)
      • Link linear velocity imitation
      • Link angular velocity imitation
    • Regularization and Penalties:
      • Action rate regularization
      • Joint limit penalties
      • Undesired contact penalties

  9. Set up VSCode for Isaac Sim indexing

    main

    To enable intelligent code suggestions and module indexing for Isaac Sim and Omniverse extensions in VSCode:

    1. Press Ctrl+Shift+P to open the Command Palette.
    2. Select Tasks: Run Task.
    3. Run setup_python_env.
    4. When prompted, provide the absolute path to your Isaac Sim installation.

    This creates a .python.env file in the .vscode directory.

  10. Install InstinctLab

    main

    To install InstinctLab, you must first have Isaac Lab (version 5.1.0) and Instinct-RL installed. Clone this repository outside of your Isaac Lab directory, then install the library using an interpreter that has Isaac Lab available.

    # 1. Clone the repository
git clone https://github.com/project-instinct/instinctlab.git

# 2. Install the library
# Ensure you are using a python interpreter with Isaac Lab installed
python -m pip install -e source/instinctlab

  11. Configure a NoisyGroupedRayCasterCamera

    main

    To use a NoisyGroupedRayCasterCameraCfg, you must define the sensor's attachment point (prim_path), the target meshes (mesh_prim_paths), the camera's physical properties (offset, rotation, pattern), and the noise pipeline.

    Note that when targeting robot links, you should use NoisyGroupedRayCasterCameraCfg.RaycastTargetCfg(prim_expr=...) within the mesh_prim_paths list to ensure the raycaster tracks the moving robot parts.

    from instinctlab.sensors import GroupedRayCasterCfg, NoisedGroupedRayCasterCameraCfg

# Configure noisy camera (Example: RealSense D435i)
noisy_camera = NoisyGroupedRayCasterCameraCfg(
    prim_path="{ENV_REGEX_NS}/Robot/torso_link",
    mesh_prim_paths=[
        "/World/ground",
        # Use RaycastTargetCfg for moving robot links
        # NoisyGroupedRayCasterCameraCfg.RaycastTargetCfg(prim_expr="/World/envs/env_.*/Robot/torso_link/visuals")
    ],
    offset=NoisyGroupedRayCasterCameraCfg.OffsetCfg(
        pos=(0.04, 0.015, 0.46),
        rot=(0.707, 0.0, 0.707, 0.0),
        convention="world",
    ),
    ray_alignment="yaw",
    pattern_cfg=patterns.PinholeCameraPatternCfg(
        focal_length=1.0,
        horizontal_aperture=1.5,
        vertical_aperture=1.0,
        height=270,
        width=480,
    ),
    data_types=["distance_to_image_plane"],
    noise_pipeline={
        "depth_artifact_noise": DepthArtifactNoiseCfg(),
        "stereo_noise": RangeBasedGaussianNoiseCfg(
            max_value=1.2,
            min_value=0.12,
            noise_std=0.02,
        ),
        "normalize": DepthNormalizationCfg(
            depth_range=(0.0, 1.5),
            normalize=True,
        ),
    },
    update_period=1 / 60,
    depth_clipping_behavior="max",
    min_distance=0.05,
    max_distance=2.0,
)

  12. Perform Whole Body Shadowing

    main

    Whole Body Shadowing (Task ID: Instinct-Shadowing-WholeBody-Plane-G1-v0) allows for shadowing using whole-body motions. You must configure the motion files in whole_body/config/g1/plane_shadowing_cfg.py.

    Configuration Keys:

    • MOTION_NAME: A unique identifier for the motion.
    • _path_: The directory path where motion files are stored.
    • _hacked_selected_files_: The specific filenames of the motion, relative to the _path_.

    Follow the standard training and playback commands via the CLI.

    # 1. Train the policy
python scripts/instinct_rl/train.py --headless --task=Instinct-Shadowing-WholeBody-Plane-G1-v0

# 2. Play the trained policy
python scripts/instinct_rl/play.py --task=Instinct-Shadowing-WholeBody-Plane-G1-v0 --load_run=<run_name>