DOCSEARCHbyAlgolia
Submit Docs

Gymnasium-Robotics Documentation

repository·main·Indexed 21 days ago

https://github.com/farama-foundation/gymnasium-robotics

A collection of robotic manipulation environments for reinforcement learning built on the Gymnasium API. It includes the GoalEnv class for multi-goal API structures supporting Hindsight Experience Replay (HER), as well as specific environment suites such as Adroit Hand, Fetch, Franka Kitchen, and MaMuJoCo (Multi-Agent MuJoCo) for agent factorization research. Requires the MuJoCo physics engine.

Tokens
18.3K
Snippets
32
Records
72
Agent score
74%

What's inside gymnasium-robotics

  1. Overview of Maze environments

    main

    The Maze environments consist of a collection of tasks where an agent must navigate through a maze to reach a specific goal position.

    There are two types of agents available for these environments:

    1. 2-DoF force-controlled ball: A ball agent with 2 degrees of freedom.
    2. Ant agent: The classic Ant agent from the Gymnasium MuJoCo environments.

    Environments can be initialized with various maze shapes that provide increasing levels of difficulty.

  2. Overview of Fetch environments

    main

    The Fetch environments are based on the 7-DoF Fetch Mobile Manipulator arm equipped with a two-fingered parallel gripper. These environments provide various manipulation tasks for reinforcement learning:

    • FetchReach-v4: Moving the end-effector to a target goal position.
    • FetchPush-v4: Pushing a box to a target goal position.
    • FetchSlide-v4: Hitting a puck across a table so it slides to a target goal.
    • FetchPickAndPlace-v4: Picking up a box from a table and moving it to a target goal position above the table.

  3. Available Adroit Hand environments

    main

    The Adroit Hand environments feature a Shadow Dexterous Hand attached to a free arm, supporting up to 30 actuated degrees of freedom. There are four primary tasks available, each with a standard version and a sparse reward variant.

    Standard Environments

    These environments provide continuous reward signals for the tasks:

    • AdroitHandDoor-v2: Opening a door with a latch.
    • AdroitHandHammer-v2: Hammering a nail into a board.
    • AdroitHandPen-v2: Manipulating a pen to a target position and rotation.
    • AdroitHandRelocate-v2: Picking up a ball and moving it to a target location.

    Sparse Reward Environments

    These variants provide a reward of 10.0 for achieving the target goal and -0.1 otherwise:

    • AdroitHandDoorSparse-v2
    • AdroitHandHammerSparse-v2
    • AdroitHandPenSparse-v2
    • AdroitHandRelocateSparse-v2

  4. Use Shadow Dexterous Hand environments

    main

    The Shadow Dexterous Hand environments are based on an anthropomorphic robotic hand with 24 degrees of freedom (20 independently controllable).

    Available base environments include:

    • HandReach-v2: Reaching a goal position with the thumb and a selected finger.
    • HandManipulateBlock-v1: Manipulating a block to a target position and rotation.
    • HandManipulateEgg-v1: Manipulating an egg to a target position and rotation.
    • HandManipulatePen-v1: Manipulating a pen to a target position and rotation.

  6. Use MaMuJoCo (Multi-Agent MuJoCo) for agent factorization research

    main

    MaMuJoCo provides a framework for researching agent factorization by creating multi-agent versions of Gymnasium/MuJoCo tasks. It supports two types of environments: (1) multi-agent factorizations of existing Gymnasium/MuJoCo tasks and (2) new complex MuJoCo tasks designed for multi-agent algorithms.

    MaMuJoCo primarily uses the PettingZoo.ParallelAPI, but also supports the PettingZoo.AECAPI.

  7. Understand Ant action space partitioning

    main

    The Ant environment supports different partitioning schemes that determine how many agents are present and how the action space is distributed across them. This allows for multi-agent training where different parts of the robot are controlled by different agents.

    Partitioning Options:

    1. None (Default):

      • Agents: 1 (agent_0)
      • Action Space: A single Box(-1, 1, (8,), float32) containing all 8 joint torques.
      • Use case: Standard single-agent control equivalent to Gymnasium's MuJoCo/Ant.

    2. "2x4" (Neighboring legs):

      • Agents: 2 (agent_0, agent_1)
      • Action Space: Each agent has a Box(-1, 1, (4,), float32).
      • Structure: agent_0 controls the front legs; agent_1 controls the back legs.

    3. "2x4d" (Diagonal legs):

      • Agents: 2 (agent_0, agent_1)
      • Action Space: Each agent has a Box(-1, 1, (4,), float32).
      • Structure: Agents control diagonal pairs of legs.

    4. "4x2" (Individual legs):

      • Agents: 4 (agent_0 through agent_3)
      • Action Space: Each agent has a Box(-1, 1, (2,), float32).
      • Structure: Each agent controls exactly one leg (hip and ankle joints).

  8. Understand the Ant observation space in MaMuJoCo

    main

    In the MaMuJoCo Ant environment, observations are split into local and global categories. Each agent receives a local observation based on its assigned parts of the agent (determined by local_categories and observation depth) and a global observation.

    In addition to local observations, every agent observes the position and velocity of the ant's torso (the root node).

    Observation Categories:

    • Default local_categories: [['qpos', 'qvel'], ['qpos']]
    • Default global_categories: ('qpos', 'qvel')
    • Supported categories: 'qpos', 'qvel', 'cfrc_ext'

    For detailed information on the underlying MuJoCo observation structure, refer to the Gymnasium Ant documentation.

    | Observation Categories ||
|-----------------------|------------------------------------------------------|
| Default `local_categories` | `[['qpos', 'qvel'], ['qpos']]` |
| Default `global_categories` | `('qpos', 'qvel')` |
| Supported observation categories | `'qpos', 'qvel', 'cfrc_ext'` |

  9. MaMuJoCo single action environment specifications

    main

    When using MaMuJoCo single action environments (instantiated with None as the partition), the following properties apply:

    • Action Space: The shape depends on the partitioning (though for single-agent mode, it matches the standard Gymnasium MuJoCo environment).
    • Observation Space: The agent receives the same observations as the single-agent Gymnasium environment.
    • Rewards: The agent receives the same reward as the single-agent Gymnasium environment.
    • Starting State: The starting state is identical to the single-agent Gymnasium environment.
    • Episode End: Termination and truncation conditions are the same as the single-agent Gymnasium environment.

  11. Rewards and Episode Termination in Humanoid Standup

    main

    The MaMuJoCo Humanoid Standup environment follows these rules for rewards and termination:

    • Rewards: All agents receive the same reward as the standard Gymnasium MuJoCo Humanoid Standup.
    • Starting State: Uses the same starting state as the standard Gymnasium MuJoCo Humanoid Standup.
    • Episode End: All agents terminate and truncate simultaneously based on the same conditions as the standard Gymnasium MuJoCo Humanoid Standup.

  12. Understand the Multi-goal API structure

    main

    Robotic environments in gymnasium-robotics extend the core Gymnasium API by inheriting from the GoalEnv class. This API requires the observation space to be a dictionary containing exactly three keys:

    • observation: The actual state observation of the environment.
    • desired_goal: The target goal the agent is tasked to achieve.
    • achieved_goal: The current state of the goal as achieved by the agent. The objective is to minimize the distance between achieved_goal and desired_goal.

    This structure is specifically designed to support algorithms like Hindsight Experience Replay (HER) by allowing rewards and termination signals to be re-computed using substituted goals.