OmniIsaacGymEnvs

Even with fixed seeds, achieving perfect determinism in GPU-accelerated simulations is challenging due to several factors:

1. GPU Work Scheduling

GPU scheduling can alter the order of operations. Because of floating-point numeric storage, changes in execution order can cause small differences in the least significant bits, which diverge over thousands of simulation frames.

2. Domain Randomization Timing

Certain parameters, such as object scales, cannot be randomized during runtime without breaking determinism or causing simulation issues. These must be set at setup time using the on_startup condition for Domain Randomization.

3. Floating Point Precision and World Origin

Environments placed far from the world origin (0, 0, 0) accumulate floating-point errors. This can cause states to differ even when the same actions are applied to the same initial states.

• Workaround: Place all actors/environments at the world origin (0, 0, 0) and filter out collisions between them. Note that this may cause a 15-50% performance degradation.

4. Resetting into Contact States

If actors are reset into a state where they are already in contact with other actors, the simulation may become non-deterministic. This is because contacts are re-computed from scratch during each reset scenario and cannot guarantee identical results across computations.

Environment wrappers provide the vectorized interface required by RL libraries to communicate with Isaac Sim.

VecEnvBase (Single-Threaded)

Standard interface for most RL libraries. It runs in the same thread as the simulation. Key APIs:

• render(mode="human"): Renders the current frame.
• seed(seed=-1): Sets the random seed.
• step(actions): Executes a physics step, computes observations/rewards/dones, and returns buffers.
• reset(): Resets the environment and re-computes observations.
• close(): Shuts down the simulator.

VecEnvMT (Multi-Threaded)

Designed to isolate the RL policy in a separate thread from the simulation and rendering. This prevents UI interactions from interfering with the training loop.

• Requirement: You must implement a TrainerMT class with a run() method to initiate the RL loop.
• Timeout: VecEnvMT has a default 90-second timeout. If the RL thread or simulation thread waits longer than this for the other, an exception is thrown. Increase this via VecEnvMT.initialize(timeout=...) for complex scenes.
• Note: Currently only supported within the extension workflow.

The extension workflow (introduced in Isaac Sim 2023.1.0) provides a UI-driven experience for running reinforcement learning environments, separating scene creation from training execution.

Key Benefits:

• Faster Iteration: You can re-create scenes or re-launch training runs without closing the Isaac Sim app.
• Hot Reloading: Changes to task code or configuration parameters are picked up when clicking the LOAD or Train buttons.
• Separation of Concerns: The UI manages scene loading, while the RL policy runs on a separate thread using a multi-threaded VecEnv base environment. Communication between the UI thread and the RL thread occurs via multi-threaded queues.

Workflow Concept:

1. Load Scene: Use the UI to select a task and click LOAD to populate the stage with assets.
2. Run RL: Click START to launch the RL thread for training or inference.
3. Iterate: Modify code/configs and click LOAD or START again to apply changes without a full app restart.

When using ArticulationView, RigidPrimView, or RigidContactView from omni.isaac.core, you must only call a specific setter API once per simulation step per view instance. Subsequent calls to the same setter API within the same step will override previous calls, effectively voiding them.

To apply changes to multiple indices or objects, you should aggregate all desired states into a single buffer and make one call to the setter API before calling my_world.step().

# INCORRECT: Subsequent calls override previous ones
my_view.set_world_poses(positions=[[0, 0, 1]], orientations=[[1, 0, 0, 0]], indices=[0])
my_view.set_world_poses(positions=[[0, 1, 1]], orientations=[[1, 0, 0, 0]], indices=[1])
my_world.step()

# CORRECT: Use a single call with combined buffers
my_view.set_world_poses(positions=[[0, 0, 1], [0, 1, 1]], orientations=[[1, 0, 0, 0], [1, 0, 0, 0]], indices=[0, 1])
my_world.step()

Domain Randomization (DR) is used to make reinforcement learning agents robust to physical variations (e.g., for sim2real transfer) by repeatedly randomizing simulation dynamics during training.

OmniverseIsaacGymEnvs supports "on the fly" domain randomization via the omni.replicator.isaac extension. This allows dynamics to change without reloading assets or re-parsing files.

Users can implement DR in two ways:

1. Python API: Directly using methods from the omni.replicator.isaac extension.
2. YAML Interface: Specifying DR settings in the task configuration .yaml file (the recommended approach for most users).

DR can be applied to five main parameter groups:

• observations: Noise added to agent observations.
• actions: Noise added to agent actions.
• simulation: Physical parameters for the entire scene (e.g., gravity).
• rigid_prim_views: Properties of rigid prims (e.g., material_properties).
• articulation_views: Properties of articulations (e.g., joint stiffness).

The RL ecosystem in omniisaacgymenvs consists of three primary components:

1. Task: The core logic layer where observations, rewards, and actions are computed. It interacts directly with the simulation actors.
2. RL Policy: The learning algorithm (e.g., PPO via rl_games) that consumes observations and produces actions.
3. Environment Wrapper: The interface layer that facilitates communication between the Task and the RL Policy. It provides a vectorized API (like gym.Env) to the policy.

Tasks are typically implemented by inheriting from RLTask, which provides common utilities for configuration parsing and buffer initialization.

class MyNewTask(RLTask):
    # Task logic (observations, rewards, etc.)
    ...

# The wrapper connects the Task to the Policy
env = VecEnvBase(headless=False)
env.set_task(my_task_instance)

# The policy interacts with the wrapper
obs, rew, done, info = env.step(actions)