ovrtx

NVIDIA ovrtx is a lightweight C and Python SDK that exposes Omniverse RTX technology. It is designed for real-time, physically accurate sensor simulation and rendering, specifically targeting robotics learning, synthetic data generation, and industrial/design workflows.

Key Features

• Physically Accurate Sensor Simulation: Simulates cameras, lidar, radar, and other sensors.
• Scalable Performance: Supports high-throughput simulation (tens of thousands of frames per second) for reinforcement learning, real-time interactive viewports, and offline predictive rendering.
• OpenUSD Integration: Uses OpenUSD for scene description, enabling interchange with CAD and simulation tools.
• Python Ecosystem Integration: Designed for easy use within Python-based simulation and machine learning workflows.

The skills/ directory contains structured SKILL.md files designed for AI coding agents. Each skill provides a self-contained reference for a specific task, including Python and C API usage, prerequisites, and troubleshooting.

Each skill is organized into its own subdirectory and follows a strict format to ensure accuracy and maintainability.

To use the ovrtx runtime, your environment must meet the following requirements:

• GPU: NVIDIA RTX-capable GPU.
• Driver: A supported NVIDIA driver (refer to ../driver_requirements for specific versions).
• Network: Internet access (the minimal example requires downloading scene assets from S3).
• Environment: Execution must occur outside of sandboxed environments.

In ovrtx 0.3.x, USD loading is determined by your intent for how the content should exist in the stage:

• Single scene / Root layer: Use open_usd() to set the active root layer.
• Runtime inline root layer (often with subLayers): Use open_usd_from_string*.
• Additive removable content: Use add_usd_reference* at a specific prim path. You must keep the returned handle to call remove_usd later.

When using precompiled CUDA artifacts, you can split a single node across multiple files by compiling them with nvcc and shipping the resulting .ptx, .cubin, or .fatbin.

Important Note on Includes: A CUDA node is treated as a single translation unit compiled against a single include directory. SPG searches for headers using the directory defined by the CUDA_PATH or CUDA_HOME environment variables, or the headers bundled with the renderer.

The directory containing your .cu source file is NOT automatically added to the include path. Therefore, #include "MyHelpers.cuh" will fail if MyHelpers.cuh is located in the same folder as your source but not in the official CUDA include paths. To resolve this, compile your helpers into the artifact using your own toolchain before handing the binary to SPG.

Minimal mode uses a simplified lighting model to maintain speed and stability:

• DistantLight: Only the first DistantLight found in the scene is respected. Shadows from this light are hard shadows.
• DomeLight: DomeLight is ignored in Minimal mode.
• Fallback: If no DistantLight is present, the mode falls back to a single camera light.

To improve visibility without adding scene lights, you can adjust the global ambient lighting attributes or disable shadows using omni:rtx:minimal:castShadows.

When working with sensor composites like Lidar, point channels are often allocated for the worst-case capacity (the maximum possible returns).

The Stride Problem

For a channel like Coordinates, the shape is typically [3, Nmax]. The data is stored with a stride where all X values come first, then all Y, then all Z. The step between elements is the allocated capacity, not the number of valid points.

The Solution: Use the Counts channel

To avoid processing invalid data or reading out of bounds, you must bound your GPU work using the Counts channel:

1. Access the Counts tensor via pc.tensors["Counts"].
2. Pass the Counts tensor to the GPU.
3. Apply the bound (the valid count) within your kernel.

Note: The .status field is unreliable for determining if data exists for the current frame; always rely on the value in the Counts channel.

Non-visual sensors in ovrtx (such as Lidar and Radar) are designed to produce structured sensor data instead of standard camera images.

Key characteristics include:

• Structured Data: They typically expose composite PointCloud RenderVars.
• Named Tensors: These RenderVars contain specific named tensors for data like coordinates, counts, intensity, RCS (Radar Cross Section), radial velocity, and time offsets.
• Material Modeling: You can use non-visual material labels to model how these sensors interact with surfaces (return behavior) independently of the visual shading used for cameras.

In this implementation, the blur radius is a USD attribute. Because inputs:radius is connected to nothing, writing a new value to the radius attribute on the Shader prim takes effect on the next render step without requiring a reset.

To ensure exactness, the system verifies that at radius 0, the weight table is {1.0} (the identity), meaning the published AOV must match the LdrColor input byte-for-byte.

# Example of writing a new radius to the Shader prim
# (Snippet context from main.py)
# [snippet:blur-write-radius]