BEHAVIOR-1K Documentation

repository·main·Indexed 23 days ago

https://github.com/stanfordvl/behavior-1k

A comprehensive simulation benchmark for embodied AI agents featuring 1,000 everyday household activities. The repository includes an asset pipeline for converting 3ds Max files to USD, documentation on the HumanFemale USD asset structure (including LOD and rig complexity), and the Behavior Domain Definition Language (BDDL) for defining activity problems, goals, and simulator integration.

Tokens
95K
Snippets
150
Records
364
Agent score
82%

What's inside BEHAVIOR-1K

  1. Overview of BEHAVIOR-1K

    main
    BEHAVIOR-1K is a simulation benchmark for testing embodied AI agents across 1,000 everyday household activities. It is designed for training and evaluating agents on human-centered tasks such as cleaning, cooking, and organizing. The activities are derived from real human time-use surveys and preference studies to ensure realistic simulation environments.
  2. Overview of OmniGibson

    main

    OmniGibson is an Embodied AI research platform built on NVIDIA's Omniverse. It provides a high-fidelity simulation environment characterized by:

    • Photorealistic Visuals and Physical Realism: High-quality rendering and physics.
    • Advanced Material Support: Support for fluid and soft body simulations.
    • Large-Scale Assets: High-quality scenes and objects.
    • Dynamic States: Kinematic and semantic object states that change dynamically.
    • Robotics: Mobile manipulator robots equipped with modular controllers.
    • Standardized Interface: An OpenAI Gym interface for easy integration with reinforcement learning workflows.
  3. Overview of available Object types in the Simulator

    main

    The simulator provides several object classes that can be imported into the environment. Most objects are initialized using a name or a file path (located within igibson.assets_path) and provide a load() function to import the object into PyBullet. This load() function is typically invoked by the Simulator via import_object or import_object methods.

    Supported object types include:

    • YCBObject
    • RBOObject
    • ShapeNetObject
    • Pedestrian
    • ArticulatedObject (provides APIs to get/set object pose)
    • URDFObject
    • SoftObject
    • Cube
    • VisualMarker
    • VisualShape
  4. Overview of OmniGibson Controller Types

    main

    OmniGibson supports six types of controllers categorized by their purpose and the robot morphology they target:

    • General Controllers: Agnostic to robot morphology; can be used on any robot.
    • Locomotion Controllers: Specifically for robots with navigation capabilities.
    • Manipulation Arm Controllers: Designed to control a robot's end-effector pose.
    • Manipulation Gripper Controllers: Designed to control a robot's gripper joints.
  5. Overview of the 2026 BEHAVIOR Challenge

    main

    The BEHAVIOR Challenge requires agents to solve 100 full-length household tasks within the BEHAVIOR-1K environment. The challenge evaluates an agent's ability to combine high-level reasoning, long-horizon navigation, and dexterous bimanual manipulation in house-scale scenes.

    Challenge Specifications

    • Tasks: 100 full-length household tasks.
    • Environments: 7 scenes (including 4 new scenes).
    • Evaluation Track: Single track using RGB + depth + proprioception.
    • Demonstrations: 20,000 human teleoperation demos (approx. 1,950 hours).
    • Baselines: π0.5 (pi0.5) and GR00T N1.7.
    • Ranking Metric: Average task success score with BDDL partial credit.

    Important Dates

    • Challenge Launch: 07/02/2026
    • Submission Deadline: 10/16/2026
    • Winners Announcement: 11/04/2026
  6. Key features and capabilities of OmniGibson

    main

    OmniGibson provides a high-level interface for robotics simulation and reinforcement learning, abstracting away the complexities of Isaac Sim.

    Core Capabilities:

    • Abstraction: Users interact with familiar scene, object, robot, and task interfaces instead of NVIDIA-specific code.
    • High-Level APIs: Fast methods for loading scenes, setting up tasks, and controlling robots.
    • BDDL Support: Implements samplers and checkers for all predicates and functions defined in the BDDL standard, enabling the simulation of BEHAVIOR-1K tasks.
    • Dataset Utilities: Tools for working with the BEHAVIOR dataset, including decryption and saving/loading scene states.
    • Vectorization: Supports simple vectorization across multiple scene copies to facilitate reinforcement learning training.
    • Robot Controllers: Includes configurable controllers for direct joint control, inverse kinematics (IK), operational space, and differential drive.
  7. Understand the HumanFemale USD asset structure

    main

    The HumanFemale asset is a demonstration of using UsdSkel schemas to describe a skeletally-deformed character in USD. It is composed of several layers that manage geometry, materials, skeleton bindings, and animation clips.

    Core Components

    • HumanFemale.usd: The primary asset file. It defines the character's geometry, materials, and UsdSkelBindingAPI properties that control how the skeleton affects the geometry.
    • assets/HumanFemale.rig.usd: Defines the character's Skeleton, including its bind pose and rest pose. It uses the rigComplexity variant set to provide different skeleton configurations.
    • Animation Clips: Files like HumanFemale.keepAlive.usd and HumanFemale.walk.usd provide animation clips containing joint transforms and root character transforms. While these clips reference the main asset for visualization, they can be authored with only joint animations.
    • Blendshapes: Blendshape data is organized by geometric LOD (low, medium, full) in files like assets/HumanFemale.{LOD}.blendshapes.usd. These are used by the 'high' complexity rig variant.
  8. Overview of the assets folder structure

    main

    The downloaded assets directory follows this structure:

    • models/: Contains robot models (e.g., ant, fetch, locobot), interactive objects, articulated objects, and scene components.
    • networks/: Contains the pretrained model.pth (the "goggle" network used in Gibson V1).
    • test/: Contains mesh files used for verifying renderer installation.
    • example_configs/: Contains sample configuration files (deprecated).
    assets
    ├── models
    │   ├── ant
    │   ├── ...
    │   └── ycb
    ├── networks
    │   └── model.pth
    ├── test
    │   └── mesh
    └── example_configs
        └── {}.yaml
  9. What is a Prim in OmniGibson?

    main

    A Prim (short for "primitive") is the fundamental building block of the Omniverse scene representation (the "stage"). It represents a single scene component such as a rigid body, joint, light, camera, or material.

    In OmniGibson, Prim classes encapsulate underlying Omniverse UsdPrim instances, providing direct access to low-level Omniverse APIs. Each Prim instance is uniquely identified by its prim_path, a filepath-like string (e.g., /World/robot0/gripper_link) that defines its name and its position in the scene hierarchy.

  10. What is BDDL (BEHAVIOR Domain Definition Language)?

    main

    BDDL is an object-centric domain-specific language based on predicate logic, designed for the BEHAVIOR benchmark for embodied AI agents. It is used to define realistic and complex activities in simulation by symbolically expressing an activity's initial and goal conditions.

    Key capabilities of the BDDL codebase include:

    • Parsing BDDL files (including custom ones).
    • Compiling symbolic definitions into grounded physical simulations.
    • Efficiently checking success and progress at every simulator step.
    • Solving goal conditions to measure fine-grained progress for evaluation.
  11. Overview of iGibson Scene Types

    main

    iGibson provides four distinct scene types depending on your requirements for complexity and interactivity:

    • EmptyScene and StadiumScene: Simple scenes with flat grounds and no obstacles. Best for debugging.
    • StaticIndoorScene: Loads static 3D scenes from igibson.g_dataset_path. It handles floor information, mesh loading, and builds internal traversability graphs for pathfinding.
    • InteractiveIndoorScene: Loads fully interactive 3D scenes from igibson.ig_dataset_path. It includes advanced features like material/texture randomization, object randomization, scene quality checks, partial scene loading (by category, room type, or instance), and APIs to change the state of articulated objects (e.g., opening fridges).

    Most scenes are loaded via a load() function, which is typically invoked by the Simulator using import_scene or import_ig_scene.