DOCSEARCHbyAlgolia
Submit Docs

RViz for ROS 2

repository·rolling·Indexed 19 days ago

https://github.com/ros2/rviz

A 3D visualization tool for ROS 2 used to visualize robot sensor data, models, and coordinate frames. It features a pluggable transformation library and an extensible system for adding custom displays, panels, tools, view controllers, and frame transformation logic. The documentation covers building from source, plugin development, visual testing using VisualTestFixture, and migration guides for porting plugins from ROS to ROS 2.

Tokens
12.2K
Snippets
20
Records
59
Agent score
66%

What's inside ros2-rviz

  1. Overview of RViz core packages

    rolling

    Understanding the package structure is essential for plugin development:

    • rviz_common: The primary package for developers. Contains base classes for panels, view controllers, displays, and tools, as well as ROS 2 access points via ros_integration and property management classes.
    • rviz_rendering: Contains all rendering-related functionality, including scene graph objects (arrows, shapes, text), the render window (RenderWindowOgreAdapter), and material management.
    • rviz_default_plugins: Contains the standard plugins shipped with RViz. Use this as a reference or to derive from existing complex plugins.
    • rviz_visual_testing_framework: Used for writing automated screenshot-based visual tests for your plugins.
    • rviz2: The main application entry point (not intended for direct plugin use).

  2. Extend RViz using plugins

    rolling

    RViz is designed to be extensible. You can add new functionality through plugins in the following categories:

    • Displays: New ways to visualize data.
    • Tools: New interaction tools for the viewport.
    • View Controllers: New ways to manipulate the camera/view.
    • Panels: New UI components (like the Time panel).

    Installed plugins are automatically indexed and made available within the RViz interface. For detailed instructions on implementing your own, refer to the official RViz tutorials on the ROS wiki.

  3. Understand Display Statuses

    rolling

    Each display indicates its health via a status in the Displays list. The top-level status is determined by the worst sub-status found within the display's properties.

    Possible statuses:

    • OK
    • Warning
    • Error
    • Disabled

    Status information is indicated by text color and icons in the display title and can be inspected by expanding the Status category within the display's properties.

  4. Extend RViz via Plugins

    rolling

    RViz is designed to be extensible through a plugin system. Developers can add new functionality at several different extension points:

    • Displays: Visual representations of data (e.g., LaserScan, PointCloud).
    • Panels: UI elements within the RViz window (e.g., Time, Tool Properties).
    • Tools: Interactive elements used by the user (e.g., 2D Nav Goal, Publish Point).
    • Frames transformation library: Custom logic for handling coordinate frame transformations.
    • View Controllers: Logic for how the camera moves and interacts with the scene.

    For detailed implementation instructions, refer to the plugin_development.md guide.

  5. Use the Time panel to monitor simulation time

    rolling

    The Time panel is primarily used when running in a simulator. It provides a comparison between ROS Time (the simulated time) and Wall Clock (real-world time).

    Key functionality:

    • Monitor Time Drift: See how much simulated time has elapsed relative to real time.
    • Reset Visualizer State: You can reset the visualizer's internal time state, which triggers a reset of all active displays and clears the tf (transform) internal data cache.

    Shortcut: Press r on your keyboard to trigger the reset functionality.

    Note: If you are not running in a simulation, the Time panel is generally unnecessary and can be closed to free up screen real estate.

  6. How visual comparison and MSE thresholds work

    rolling

    The framework performs pixelwise comparison between test images and reference images. Because screenshots vary across different systems, it uses the Mean Square Error (MSE) index to determine if a test passes.

    MSE Logic

    • MSE Value: A value between 0 (identical) and 1 (completely different).
    • Default Threshold: 0.01. A test passes if the computed MSE is lower than this threshold.
    • Comparison Result: If the images are not identical, a difference image is generated where each pixel's color represents the absolute difference between the test and reference pixels.

    Customizing Thresholds

    You can set a specific threshold for an individual test using the setTesterThreshold(double threshold) method on a VisualTestFixture object. This is useful for tests that are naturally more sensitive to small fluctuations.

    // Example of setting a custom threshold in a test
fixture.setTesterThreshold(0.05);
fixture.assertMainWindowIdentity();

  7. Automate GUI interaction for visual tests

    rolling

    To prepare a 3D scene (e.g., adding displays or modifying properties) before taking a screenshot, the framework provides a hierarchy of classes to interact with the RViz GUI via QTest.

    Core Classes

    • BasePageObject: The base class for all display page objects. It provides methods to modify basic properties like QString, boolean, and QComboBox.
    • <DisplayName>DisplayPageObject: Specialized classes (e.g., PointCloudDisplayPageObject) that inherit from BasePageObject and implement methods specific to that display type.
    • DisplayHandler: Provides methods to add or remove displays from the scene.
    • TestFixture: The main test environment providing the necessary functionality to orchestrate tests.

  8. Restrict a display to a specific transformation plugin

    rolling

    If your display requires a specific transformation implementation (e.g., it only works with tf2), use the rviz_default_plugins::transformation::TransformerGuard class.

    Implementation Steps:

    1. Add a templated instance of TransformerGuard as a member variable in your display class.
    2. Initialize the guard in your display's constructor, passing a raw pointer to the display and the name of the required transformer.
    3. In your display's onInitialize() method, call TransformerGuard::initialize(rviz_common::DisplayContext * context), passing the context_ member of your display.

    Capabilities:

    • The guard will automatically disable the display and show an error status if the current transformer is incompatible.
    • Use TransformerGuard::checkTransformer() to return a boolean indicating if the current transformer is of the allowed type, allowing your display to change behavior dynamically.

  9. How the pluggable transformation library works

    rolling

    Unlike RViz for ROS 1 which was hardcoded to use tf2, RViz for ROS 2 uses a pluggable transformation library. This allows users to dynamically load and change different transformation library plugins via the Transformation panel in the GUI.

    Available Bundled Plugins

    • TFFrameTransformer (in rviz_default_plugins): Provides standard tf2 functionality; used by default.
    • IdentityFrameTransformer (in rviz_common): Performs identity transforms; used as a fallback if tf2 is unavailable.

    Developing for Transformations

    When creating custom displays that depend on specific transformation behaviors, use the TransformerGuard class. Adding this class to a display ensures the display is automatically disabled if an incompatible transformer is selected in the GUI.

  10. How the RViz Visual Testing execution works

    rolling

    Visual tests require a running instance of RViz (VisualizerApp) and a QApplication. Because QApplication::exec() enters a blocking main event loop, standard synchronous code cannot interact with the GUI.

    The Execution Model:

    1. Asynchronous Interaction: The framework uses QTimer::singleShot to register test actions (like setting properties or moving the camera) to be executed after a specific delay. This ensures actions occur while the QApplication is running in its main event loop.
    2. The Executor: The Executor class handles the scheduling of these delayed actions so the user can write tests in a seemingly synchronous style.
    3. Lifecycle:
      • A custom RViz configuration is loaded at startup (empty scene, help panel hidden).
      • The test performs interactions via the Executor.
      • The test ends by calling an assertion method (assertMainWindowIdentity or assertScreenShotsIdentity), which triggers the final screenshot capture and comparison.
      • After each test, the scene is cleaned, the application is reset, and all displays are removed before the next test begins.

  11. Available RViz Plugin Extension Points

    rolling

    RViz can be extended at several different points. To be recognized by RViz, every plugin must derive from the specific base type listed below:

    plugin typebase type
    Displayrviz_common::Display
    Panelrviz_common::Panel
    Toolrviz_common::Tool
    Frames transformation libraryrviz_common::transformation::FrameTransformer
    View Controllerrviz_common::ViewController

  12. Achieve stable visual tests using MSE thresholds

    rolling

    Visual tests are sensitive to hardware (GPU, screen resolution, etc.). To prevent false negatives caused by pixel-wise differences, the framework uses the Mean Squared Error (MSE) index.

    Best Practices for Stability:

    • Focus on Large Objects: Tests should focus on one or a few large objects that occupy most of the scene. Small details (like individual points in a pointcloud) are unreliable for visual testing.
    • Set Appropriate Thresholds:
      • The default MSE threshold is 0.01.
      • A failing test (e.g., object not rendered or wrong color) typically produces an MSE of ~0.1 or higher.
      • A successful test on a different machine typically produces an MSE of ~0.001.
      • Use setTesterThreshold(double threshold) to adjust this value based on your specific test requirements.
    • Avoid Small Details: Do not attempt to test the presence of tiny objects, as they may not render consistently across different machines.