DOCSEARCHbyAlgolia
Submit Docs

ZED ROS 2 Wrapper

repository·master·Indexed 43 days ago

https://github.com/stereolabs/zed-ros2-wrapper

ROS 2 packages to interface with Stereolabs ZED cameras, exposing depth, point clouds, object detection, and human skeleton data to the ROS 2 ecosystem. Supports ROS 2 distributions including Humble, Jazzy, Lyrical, Foxy, and Rolling across Ubuntu and NVIDIA Jetson platforms. Includes tools for building Docker images and configuring camera models such as ZED, ZED 2i, and ZED X.

Tokens
7.5K
Snippets
17
Records
37
Agent score
55%

What's inside ZED ROS 2 Wrapper

  2. Configure GNSS Fusion for robot localization

    master

    The wrapper can fuse GNSS data with Positional Tracking to obtain Earth-coordinate localization.

    Requirements:

    • Subscribe to a NavSatFix topic.
    • Set gnss_fusion.gnss_fusion_enabled to true.
    • Set the correct gnss_frame parameter (e.g., gnss_frame:='gnss_link') when launching.

    Coordinate Conversion: Use the toLL and fromLL services to convert between Latitude/Longitude and robot map coordinates.

    Note: This feature is incompatible with the ZED X One camera.

  3. Understand ROS 2 installation modes (APT vs Source)

    master

    The Docker build process automatically decides how to install ROS 2 based on the base image's Ubuntu codename:

    1. APT Mode: Used when binary packages exist for the selected ROS 2 distribution on the base image's Ubuntu release.
    2. From-Source Mode: Used when binary packages are unavailable (e.g., running Jazzy on Ubuntu 22.04).

    Warning: Building ROS 2 from source is a heavy, best-effort process that compiles the entire ROS 2 core and can take a significant amount of time (e.g., ~2h 15min for Jazzy on x86_64 QEMU).

  5. Cross-compile Jetson (arm64) images on x86_64 desktops

    master

    You can build Jetson (arm64) images on an x86_64 host using QEMU emulation.

    Steps:

    1. Register QEMU binfmt handlers (required once per host/reboot): docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
    2. (Optional) Verify emulation works by running an arm64 container and checking uname -m.
    3. Build the image by exporting DOCKER_DEFAULT_PLATFORM=linux/arm64 before running the build script.

    Performance Warning: Building ROS 2 from source under emulation is extremely slow (can take several hours). To achieve faster builds (~15 minutes), choose a ROS 2 distribution whose target Ubuntu version matches the base image so that APT binaries can be used instead of compiling from source.

    # 1. Register QEMU handlers
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

# 2. Build Jetson image via emulation
DOCKER_DEFAULT_PLATFORM=linux/arm64 \
  ./build_jetson.sh --ros-distro lyrical --os jp7.1.0

  6. Run the ZED Docker container

    master

    To run the ZED ROS 2 wrapper in Docker, several host configurations are required for GPU access, networking, and display.

    Prerequisites

    • NVIDIA Runtime: Install the nvidia container runtime. Use --runtime nvidia and --gpus all (or -e NVIDIA_DRIVER_CAPABILITIES=all).
    • Privileged Mode: Use --privileged to allow access to hardware.
    • Display: For CUDA-based applications, share the host display using -e DISPLAY=$DISPLAY and mount /tmp/.X11-unix/:/tmp/.X11-unix. Run sudo xhost +si:localuser:root on the host first.
    • Networking/IPC: Use --network=host, --ipc=host, and --pid=host for ROS 2 functionality.

    Recommended Volume Mounts

    • /dev:/dev: Access video and other devices.
    • /dev/shm:/dev/shm: Required for ROS 2 shared memory.
    • /usr/local/zed/settings: Persist camera calibration files.
    • /usr/local/zed/resources: Persist AI models (Object Detection, Skeleton Tracking, etc.).

    Camera Specifics

    • USB3 Cameras: Standard mounts for /dev, /dev/shm, and ZED resources.
    • GMSL2 Cameras (ZED X, ZED X One): Requires additional mounts for /tmp, /var/nvidia/nvcam/settings/, and the zed_x_daemon.service.
    # Allow container to access EGL display resources
sudo xhost +si:localuser:root

# Run for USB3 cameras
docker run --runtime nvidia -it --privileged --network=host --ipc=host --pid=host \
  -e NVIDIA_DRIVER_CAPABILITIES=all -e DISPLAY=$DISPLAY \
  -v /tmp/.X11-unix/:/tmp/.X11-unix \
  -v /dev:/dev \
  -v /dev/shm:/dev/shm \
  -v /usr/local/zed/resources/:/usr/local/zed/resources/ \
  -v /usr/local/zed/settings/:/usr/local/zed/settings/ \
  <docker_image_tag>

# Run for GMSL cameras
docker run --runtime nvidia -it --privileged --network=host --ipc=host --pid=host \
  -e NVIDIA_DRIVER_CAPABILITIES=all -e DISPLAY=$DISPLAY \
  -v /tmp:/tmp \
  -v /dev:/dev \
  -v /var/nvidia/nvcam/settings/:/var/nvidia/nvcam/settings/ \
  -v /etc/systemd/system/zed_x_daemon.service:/etc/systemd/system/zed_x_daemon.service \
  -v /usr/local/zed/resources/:/usr/local/zed/resources/ \
  -v /usr/local/zed/settings/:/usr/local/zed/settings/ \
  <docker_image_tag>

  7. Build Desktop Docker images

    master

    Use build_desktop.sh to create images for desktop/dGPU environments. The script uses the StereoLabs gl-devel base variant.

    Default configuration:

    • ROS 2: jazzy
    • OS: ubuntu-24.04
    • ZED SDK: latest
    • CUDA: 12.8

    Available Arguments:

    • --ros-distro <distro>: Target ROS 2 distribution (e.g., humble, jazzy, lyrical).
    • --os <ubuntu-<XX.XX>>: Target Ubuntu version.
    • --sdk <X.Y.Z|latest>: Specific ZED SDK version or latest.
    • --cuda <XX.X>: Specific CUDA version.
    • --sdk-url <URL>: URL to a custom ZED SDK .run installer.
    # Defaults: ROS 2 Jazzy, Ubuntu 24.04, latest ZED SDK, CUDA 12.8
./build_desktop.sh

# ROS 2 Humble on Ubuntu 22.04 with a specific ZED SDK
./build_desktop.sh --ros-distro humble --os ubuntu-22.04 --sdk 5.4.1

  8. Debug ZED ROS2 nodes with Valgrind

    master

    To check for memory leaks and memory-related issues, run the ZED ROS2 nodes with valgrind by passing the Valgrind command as the cmd_prefix in the zed_camera_debug.launch.py launch file.

    ros2 launch zed_debug zed_camera_debug.launch.py camera_model:=<your_camera_model> cmd_prefix:='valgrind --leak-check=full --track-origins=yes'

  9. Start the ZED node

    master

    Launch the ZED node using the ros2 launch command. You must specify the camera_model parameter. The launch script uses manual composition and loads model-specific parameters from YAML files located in zed_wrapper/config.

    Available Camera Models: 'zed', 'zedm', 'zed2', 'zed2i', 'zedx', 'zedxm', 'zedxnano', 'zedxhdrmini', 'zedxhdr', 'zedxhdrmax', 'virtual', 'zedxonegs', 'zedxone4k', 'zedxonehdr'.

    To see all available launch parameters, use the -s flag.

    ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera_model>

  10. Use Simulation Mode

    master

    The ZED ROS 2 wrapper supports a simulation mode to run the ZED node with simulated data (e.g., from NVIDIA Omniverse Isaac Sim). This feature is incompatible with ZED X One and first-generation ZED cameras.

    Launch Command:

    ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera_model> sim_mode:=true

    Parameters:

    • camera_model (Mandatory): The model of the simulated camera (e.g., zedx).
    • sim_mode (Mandatory): Set to true to enable simulation.
    • use_sim_time (Optional): If true, the node waits for messages on the /clock topic and uses the simulation clock.
    • sim_address (Optional): The address of the simulation server (default: 127.0.0.1).
    • sim_port (Optional): The port of the simulation server. Must match the Streaming Port of the ZED camera streamer Action Graph node.
    ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedx sim_mode:=true

  11. Build Jetson Docker images

    master

    Use build_jetson.sh to create images for NVIDIA Jetson platforms. The script uses the StereoLabs devel base variant.

    Default configuration:

    • ROS 2: humble
    • OS: jp6.2.2 (JetPack 6.2.2 / L4T r36.5 / Ubuntu 22.04)
    • ZED SDK: latest

    Available Arguments:

    • --ros-distro <distro>: Target ROS 2 distribution (e.g., humble, jazzy, lyrical).
    • --os <jpX.Y.Z|l4t-rXX.X>: Target JetPack version or L4T release.
    • --sdk <X.Y.Z|latest>: Specific ZED SDK version or latest.
    • --sdk-url <URL>: URL to a custom ZED SDK .run installer.
    # Default: ROS 2 Humble on JetPack 6.2.2 (Ubuntu 22.04), latest ZED SDK
./build_jetson.sh

# ROS 2 Lyrical on JetPack 7.1 (Ubuntu 24.04) — built from source
./build_jetson.sh --ros-distro lyrical --os jp7.1.0

  12. Build Docker images for ZED ROS 2 Wrapper

    master

    The docker/ directory contains scripts to build Docker images for the ZED ROS 2 Wrapper. These images are built on top of official StereoLabs ZED base images (which include CUDA and the ZED SDK) and add ROS 2 and the compiled wrapper.

    There are two primary entry-point scripts depending on your hardware:

    • build_desktop.sh: For desktop/dGPU systems (uses gl-devel base).
    • build_jetson.sh: For NVIDIA Jetson systems (uses devel base).

    Before building, ensure you have checked out the desired version of the wrapper:

    git checkout <branch_or_tag>
    git checkout <branch_or_tag>