DOCSEARCHbyAlgolia
Submit Docs

iGibson 2.2.2

repository·master·Indexed 21 days ago

https://github.com/stanfordvl/igibson

A high-fidelity simulation environment for training robotic agents in large-scale, interactive 3D scenes. It integrates Bullet physics and advanced rendering to support manipulation and navigation tasks. The platform includes tools for robot control, sensor modalities (RGB, depth, LiDAR), and scene management, as well as pipelines for converting Matterport scans and custom .obj models into iGibson format using Blender 2.82.

Tokens
29.2K
Snippets
72
Records
119
Agent score
74%

What's inside iGibson

  1. Overview of iGibson Simulation Environment

    master

    iGibson (the Interactive Gibson Environment) is a simulation platform designed for training and evaluating robotic agents in large-scale, realistic indoor scenes.

    Key Features:

    • Fast Rendering & Physics: Utilizes fast visual rendering and Bullet-based physics simulation.
    • Large-Scale Datasets: Includes hundreds of 3D environments reconstructed from real homes and offices.
    • Interactive Objects: Supports objects that can be pushed, actuated, or undergo state changes (e.g., cooking, slicing, freezing in iGibson 2.0).
    • Task Support: Enables research into indoor navigation and mobile manipulation tasks like opening doors, picking/placing objects, and object searching.
    • BEHAVIOR Benchmark Compatibility: Implements features required for the BEHAVIOR benchmark, including sampling logic for activity descriptions, state checking, and connection to the BEHAVIOR 3D object dataset.

  2. Overview of available Object types in iGibson

    master

    iGibson provides several specialized object classes that can be imported into the Simulator. Most objects are initialized using a name or a path located within igibson.assets_path and utilize a load function to import the model into PyBullet.

    Supported object types include:

    • YCBObject
    • RBOObject
    • ShapeNetObject
    • Pedestrian
    • ArticulatedObject (provides APIs to get and set object pose)
    • URDFObject
    • SoftObject
    • Cube
    • VisualMarker

  3. Available motion planners in igibson.external.motion

    master

    The igibson.external.motion module provides Python implementations of various robotic motion planners. These are categorized into sampling-based planners and grid search planners.

    ### Sampling-based:
* Probabilistic Roadmap (PRM)
* Rapidly-Exploring Random Tree (RRT)
* RRT-Connect (BiRRT)
* Linear Shortcutting
* MultiRRT
* RRT*

### Grid search:
* Breadth-First Search (BFS)
* A*

  4. Understand the iG Dataset content

    master

    The iG Dataset v1.0 provides a large-scale simulation environment for robot training, consisting of:

    • 15 Large Scenes: Real-world home reconstructions converted into interactive environments. Each scene represents one floor and includes:
      • Bounding box annotations for furniture (cabinets, doors, tables, etc.).
      • Layout information (occupancy and semantics).
      • Manually designed lighting and baked textures for building elements.
      • Scene definitions using iGSDF (an extension of URDF).
    • 500+ Object Models: Cleaned models from datasets like ShapeNet and PartNet-Mobility. They include:
      • Physics-based material information (diffuse, roughness, metallic, normal).
      • Dynamics properties (weight, friction).
      • Articulation support for moving parts.
      • Definitions using URDF and OBJ files.

  5. Overview of the iGibson Physics Engine

    master

    iGibson uses PyBullet as its underlying physics engine to simulate rigid body collisions and joint actuation for robots and articulated objects.

    Because iGibson uses MeshRenderer for rendering and PyBullet for physics, the two systems must remain synchronized. While iGibson handles this synchronization internally in its high-level API, developers interacting directly with PyBullet within the iGibson environment should be aware of the following common PyBullet operations used for scene management:

    • Loading: p.createMultiBody and p.loadURDF are used to load scenes, objects, and robots.
    • Pose Resetting: p.resetBasePositionAndOrientation sets the base pose of robots and objects.
    • Joint Control: p.resetJointState sets joint positions, and p.setJointMotorControl2 is used to control robots and articulated objects.

  6. What is a BehaviorRobot?

    master

    The BehaviorRobot is a specialized embodiment designed for Virtual Reality (VR) avatars or autonomous agents participating in the BEHAVIOR100 challenge. It consists of a torso, a head link, and two hands, connected by floating joints.

    Its action space consists of 26 Degrees of Freedom (DoF):

    • Torso: 6 DoF delta pose (relative to previous frame).
    • Head: 6 DoF delta pose (relative to the new torso frame).
    • Left/Right Hand: 6 DoF delta pose (relative to the new torso frame).
    • Grasping (Left/Right): Delta change in grasping fraction (0 = open, 1 = closed).

  7. Understand Extended States in iGibson 2.0

    master

    iGibson 2.0 is an object-oriented simulator that maintains properties beyond standard kinematic data (pose, velocity, etc.). These additional properties are called extended states.

    Available extended states include:

    • Temperature: A continuous value that changes based on proximity to active heat sources or sinks.
    • Wetness level: An integer value that increases when an object contacts a water droplet.
    • Cleanliness (Dustiness and Stain Level): Represented by the number of particles on a surface. Levels decrease as particles are removed, reaching 0% when fully cleaned.
    • Toggled State: A binary functional state for objects that can be toggled on or off.
    • Sliced State: Indicates if an object (e.g., food) has been sliced into two halves.

  8. Transform 3D points from OpenGL to World Frame

    master

    When using the 3d mode, the renderer provides 4-channeled images where the first three channels are $(x, y, z)$ coordinates in the OpenGL frame. Because iGibson uses OpenGL, the coordinate system differs from the standard camera frame.

    To transform a point from the image (OpenGL frame) to the world frame, follow these steps:

    1. Convert the point from the OpenGL frame to the camera frame using a specific rotation matrix.
    2. Use the robot's 'eyes' link pose to find the camera's position and orientation in the world frame.
    3. Calculate the camera's pose relative to the robot frame.
    4. Chain the transformations: OpenGL -> Camera -> Robot -> World.
    # 1. Get camera pose in world frame from robot 'eyes' link
eye_pos, eye_orn = self.robot.links["eyes"].get_position_orientation()
camera_in_wf = quat2rotmat(xyzw2wxyz(eye_orn))
camera_in_wf[:3,3] = eye_pos

# 2. Transformation from OpenGL frame to camera frame
camera_in_openglf = quat2rotmat(euler2quat(np.pi / 2.0, 0, -np.pi / 2.0))

# 3. Get robot pose in world frame
robot_pos, robot_orn = self.robot.get_position_orientation()
robot_in_wf = quat2rotmat(xyzw2wxyz(robot_orn))
robot_in_wf[:3, 3] = robot_pos

# 4. Get camera pose in robot frame
cam_in_robot_frame = np.dot(np.linalg.inv(robot_in_wf), camera_in_wf)

# Transformation chain for a pixel [u, v]
[td_image] = self.env.simulator.renderer.render(modes=('3d'))
point_in_openglf = td_image[u, v]
point_in_cf = np.dot(camera_in_openglf, point_in_openglf)
point_in_rf = np.dot(cam_in_robot_frame, point_in_cf)
point_in_wf = np.dot(robot_in_wf, point_in_rf)

  9. Understand the different types of iGibson scenes

    master

    iGibson provides four primary scene types depending on your simulation needs:

    1. EmptyScene and StadiumScene: Simple scenes with flat grounds and no obstacles. Best used for debugging.
    2. StaticIndoorScene: Loads static 3D scenes from igibson.g_dataset_path. It handles floor information, loads meshes into PyBullet, and builds internal traversability graphs for each floor. It supports sampling random locations and computing shortest paths.
    3. InteractiveIndoorScene: Loads fully interactive 3D scenes from igibson.ig_dataset_path. In addition to static scene features, it supports:
      • Material/Texture Randomization: Randomizing materials, textures, and dynamic properties of object models.
      • Object Randomization: Randomizing object models while keeping poses and categories intact.
      • Scene Quality Checks: Verifying collision-free object models and articulated joint ranges.
      • Partial Scene Loading: Loading only specific object categories, room types, or room instances.
      • State Manipulation: Changing the state of articulated objects (e.g., opening fridges or ovens).

  10. Integrate iGibson with learning frameworks via OpenAI Gym

    master
    iGibson is designed to be compatible with any learning framework that supports the OpenAI Gym interface. This allows you to use standard reinforcement learning libraries (like TF-Agents, Stable Baselines, etc.) by treating the iGibson environment as a standard Gym environment.

  11. Use the pretrained "goggle" network

    master

    The networks/model.pth file contains a pretrained network used to fix artifacts caused by imperfect reconstruction (the "goggle" network from Gibson V1).

    Note on usage: In the current version of iGibson, mesh rendering provides photorealistic visuals. It is recommended not to use this network to maintain a higher framerate, unless specifically required for your use case.

  12. Implement a custom Task

    master

    A Task defines what an agent must achieve. To implement a custom task, your class must implement the following four methods:

    • reset_scene: Called during env.reset(). Should handle task-specific scene resets.
    • reset_agent: Called during env.reset(). Should handle task-specific agent resets.
    • step: Called during env.step(). Should handle task-specific logic for every timestep.
    • get_task_obs: Returns task-specific (non-sensory) observations, such as goal information or proprioceptive states, as a numpy array.

    Each Task must also include a list of Reward Functions (to calculate task.get_reward) and Termination Conditions (to check task.get_termination).