DOCSEARCHbyAlgolia
Submit Docs

robosuite

repository·master·Indexed 25 days ago

https://github.com/arise-initiative/robosuite

A modular simulation framework built on MuJoCo for robot learning research. It provides standardized benchmarking environments, procedural generation tools, and support for various robot controllers (OSC_POSE, JOINT_VELOCITY) and sensors. The framework includes utilities for collecting and replaying human demonstrations in HDF5 format, visual and dynamics domain randomization via Modder classes, and a flexible Observable API for simulating realistic sensor noise and latency.

Tokens
26.3K
Snippets
29
Records
134
Agent score
79%

What's inside robosuite

  1. What is robosuite?

    master
    robosuite is a simulation framework powered by the MuJoCo physics engine designed for robot learning. It provides a modular environment for designing new robot simulations, standardized benchmarking tasks for reinforcement learning (RL), and high-quality robot controllers. It is specifically built to support data-driven algorithms like reinforcement learning and imitation learning by offering reproducible environments and diverse robot embodiments (including humanoids).

  2. Overview of the robosuite simulation framework

    master

    robosuite is a modular simulation framework powered by the MuJoCo physics engine, designed specifically for robot learning. It provides a standardized set of benchmarking tasks and a flexible API for creating new robot simulation environments.

    Key capabilities include:

    • Standardized Tasks: Diverse manipulation tasks for reproducible RL benchmarking.
    • Procedural Generation: APIs to programmatically combine robot models, arenas, and parameterized 3D objects.
    • Robot Controllers: Support for joint-space velocity control, inverse kinematics (IK) control, operational space control, and whole-body control.
    • Teleoperation: Support for keyboard, spacemouse, and MuJoCo viewer drag-and-drop.
    • Multi-modal Sensors: Access to physical states, RGB cameras, depth maps, and proprioception.
    • Human Demonstrations: Utilities for collecting and replaying demonstration data (often used with the sister project robomimic).
    • Photorealistic Rendering: Support for advanced graphics, including NVIDIA Isaac Sim rendering.

  3. Use primitive object models in robosuite

    master

    The robosuite.models.objects.primitive package provides standard geometric primitive object models for use in simulations. These models are organized into submodules based on their geometric shape. You can use these primitives to instantiate simple, well-defined objects in your environments.

    Available primitive shapes include:

    • Ball: Spherical objects.
    • Box: Rectangular cuboid objects.
    • Capsule: Capsule-shaped objects.
    • Cylinder: Cylindrical objects.

  4. Understand the robosuite.models package structure

    master

    The robosuite.models package is the core organizational unit for defining the physical and logical components of a simulation environment. It is structured into several specialized subpackages that allow you to compose complex environments by combining robots, objects, grippers, and task definitions.

    Key subpackages include:

    • robosuite.models.robots: Definitions for robotic manipulators.
    • robosuite.models.grippers: Definitions for end-effectors and grasping mechanisms.
    • robosuite.models.objects: Definitions for interactive objects within the scene.
    • robosuite.models.arenas: Definitions for the physical space or environment containers.
    • robosuite.models.tasks: Definitions for the specific goals and logic of a simulation task.
    • robosuite.models.bases: Base classes and abstractions used across the models package.

  5. Available manipulation environments in robosuite

    master

    The robosuite.environments.manipulation package provides a variety of specialized manipulation tasks for robotic agents. These environments are organized into submodules based on the specific type of manipulation task being performed.

    Available task modules include:

    • Single-arm tasks:

      • door: Door opening tasks.
      • lift: Lifting objects.
      • manipulation_env: General manipulation environments.
      • nut_assembly: Nut and bolt assembly tasks.
      • pick_place: Standard pick-and-place tasks.
      • stack: Stacking objects.
      • tool_hang: Hanging tools.
      • wipe: Wiping surfaces.

    • Two-arm tasks:

      • two_arm_env: General two-arm manipulation environments.
      • two_arm_handover: Handover tasks between two arms.
      • two_arm_lift: Lifting objects using two arms.
      • two_arm_peg_in_hole: Peg-in-hole assembly using two arms.
      • two_arm_transport: Transporting objects using two arms.

  6. Explore robosuite controller parts subpackages

    master

    The robosuite.controllers.parts package provides specialized controller components for different robot subsystems. Depending on your robot configuration, you may need to interact with one of the following subpackages:

    • robosuite.controllers.parts.arm: Controllers specifically for robotic arms.
    • robosuite.controllers.parts.gripper: Controllers for end-effectors/grippers.
    • robosuite.controllers.parts.mobile_base: Controllers for mobile robot bases.
    • robosuite.controllers.parts.generic: Generic controller parts that can be applied to various subsystems.

  7. Available arena models in robosuite.models.arenas

    master

    The robosuite.models.arenas package provides various arena configurations that define the physical environment and workspace for robotic tasks. Depending on the task requirements, you can choose from the following specialized arena modules:

    • arena.Arena: The base arena class.
    • bins_arena.BinsArena: An arena containing bins.
    • empty_arena.EmptyArena: A minimal arena with no objects or surfaces.
    • multi_table_arena.MultiTableArena: An arena featuring multiple tables.
    • pegs_arena.PegsArena: An arena containing pegs (often used for insertion tasks).
    • table_arena.TableArena: A standard arena with a single table surface.
    • wipe_arena.WipeArena: An arena specifically configured for wiping tasks.

  8. Understand the robosuite.models.objects package structure

    master

    The robosuite.models.objects package is the central location for defining and managing physical objects within simulation environments. It is organized into several subpackages and modules based on the complexity and type of the object being modeled:

    Subpackages

    • composite: For objects composed of multiple parts.
    • composite_body: For objects with complex body structures.
    • group: For managing collections of objects.
    • primitive: For basic geometric shapes (e.g., spheres, cubes).

    Key Modules

    • generated_objects: Likely contains objects procedurally or programmatically generated.
    • object_groups: Provides functionality to handle groups of objects as single entities.
    • objects: The core module for defining individual object models.
    • xml_objects: Handles objects defined via XML (likely for MuJoCo URDF/MJCF integration).

  9. Available manipulator robot models in robosuite

    master

    The robosuite.models.robots.manipulators package provides various robot models designed for manipulation tasks. These models are organized into submodules, each representing a specific robot or a category of robot models. You can use these modules to instantiate specific robot configurations within your simulation environments.

    Supported robot models include:

    • Baxter: baxter_robot
    • GR1: gr1_robot
    • Humanoid: humanoid_model and humanoid_upperbody_model
    • IIWA: iiwa_robot
    • Jaco: jaco_robot
    • Kinova3: kinova3_robot
    • Legged Manipulator: legged_manipulator_model
    • Panda: panda_robot
    • Sawyer: sawyer_robot
    • Spot Arm: spot_arm
    • Tiago: tiago_robot
    • UR5e: ur5e_robot
    • Generic Manipulator: manipulator_model

  11. Understand Controller Execution and Frequency

    master

    Controllers in robosuite operate between a simulated policy frequency ($f_p$) and a simulation frequency ($f_s$).

    1. Policy Step: The policy/solution provides a desired configuration (reference value) at frequency $f_p$ (e.g., 20Hz or 30Hz).
    2. Controller Iterations: The controller executes several iterations at the simulation frequency $f_s$ (where $f_s = N \cdot f_p$) using the same reference signal provided by the policy.
    3. Error Dynamics: During these $N$ iterations, while the reference signal remains constant, the robot's state changes, causing the error between the current state and the desired configuration to evolve.
    4. Torque Output: The controller converts the reference signal and the current state into joint torques to minimize the error.

  12. How the Environment and Controllers interface with Robots

    master

    The Environment object provides OpenAI Gym-style APIs to interact with the simulation.

    Action Spaces and Controllers: Actions are sent to the Controllers of the Robots. The structure of the action space depends on the type of controller used:

    • Joint-space controllers: The action space corresponds to the number of joints in the robot.
    • Operational space controllers: The action space corresponds to 3D Cartesian movement or 6D pose of the end-effector.

    Controllers interpret these high-level action commands and transform them into low-level torques for the physics engine.

    Observations and Sensors: Sensors retrieve data from the underlying MuJoCo simulation to generate observations. Supported modalities include:

    • RGB-D cameras
    • Force-torque measurements
    • Proprioceptive data

    Environments also provide task-specific metadata, such as reward functions for reinforcement learning and success conditions.