DOCSEARCHbyAlgolia
Submit Docs

Foxglove SDK

repository·main·Indexed 43 days ago

https://github.com/foxglove/foxglove-sdk

A multi-language toolkit for logging and streaming multimodal data to the Foxglove visualization platform. It supports MCAP formats and various serialization protocols, providing a C library (wrapper around Rust), a high-level C++ SDK, and a Python-based SDK Playground. Features include remote access for live visualization and teleoperation via the RemoteAccessGateway class, as well as tools for implementing custom Foxglove Data Loaders using WASI SDK.

Tokens
89.9K
Snippets
284
Records
451
Agent score
55%

What's inside Foxglove SDK

  1. Overview of the Foxglove SDK

    main

    The Foxglove SDK enables logging and visualization of multimodal data within the Foxglove platform. It provides several core capabilities:

    • Live Streaming: Stream live data to Foxglove via a local WebSocket.
    • Data Logging: Log data to MCAP files for later analysis or visualization.
    • Message Support: Use built-in Foxglove message types for standard visualizations, or define custom messages using supported serialization formats.
    • ROS Integration: Native support for ROS via specialized packages.

    The core SDK is implemented in Rust, but prebuilt libraries and wheels are provided for Python and C++ so a Rust development environment is not required.

  2. Overview of ROS packages for Foxglove

    main

    The ROS packages in this repository allow ROS 2 users to connect their ROS middleware systems to Foxglove using either the Foxglove WebSocket or the remote access gateway.

    Architecture

    The ROS 2 implementation consists of two packages:

    1. foxglove_msgs: Contains messages useful for publishing or subscribing from Foxglove. It has no direct dependencies on other packages.
    2. foxglove_bridge: A standalone ROS 2 bridge that manages its own SDK consumption (WebSocket server and remote access gateway) and implements ROS 2-specific logic.

    Note on ROS 1: The ROS 1 bridge is maintained in a separate repository (foxglove_bridge_ros1) and is a parallel implementation that does not share code with the ROS 2 bridge.

  3. Overview of the Foxglove Rust SDK

    main

    The Foxglove Rust SDK is the official crate for integrating with the Foxglove platform. It enables developers to log events in two primary ways:

    1. Local MCAP files: Writing event data directly to MCAP files for later playback.
    2. Local visualization server: Communicating with a local server that interfaces with the Foxglove app for real-time visualization.

    For detailed API documentation, visit docs.rs/foxglove.

  4. What is foxglove_bridge?

    main

    foxglove_bridge is a high-performance ROS 2 WebSocket bridge written in C++. It uses the Foxglove SDK to provide a protocol similar to rosbridge but optimized for high-frequency topics and large messages.

    Key features include:

    • Support for ROS 2 .msg and .idl schema formats.
    • Parameter interaction.
    • Graph introspection (viewing publishers and subscribers).
    • Support for non-ROS systems.

    Note: This version of foxglove_bridge does not support ROS 1. For ROS 1, use the legacy version found in the ROS 1 foxglove_bridge repo.

  5. Understand the purpose and execution environment of container scripts

    main

    The scripts in this directory are utility scripts designed to run inside Docker containers (such as the netem sidecar). They are not intended to be executed directly on your host machine.

    Key characteristics:

    • Runtime: Written in Python because the sidecar containers use the python:3-alpine base image.
    • Dependencies: The iproute2 package (providing the tc command) is installed at container startup.
    • Deployment: These scripts are mounted into containers using Docker Compose volume binds.

  7. How Sinks, Channels, and Contexts work together

    main

    To record messages in Foxglove, you must use a combination of a Sink and a Channel, both of which are bound together by a Context.

    • Context: Acts as the binding between channels and sinks. Every channel and sink belongs to exactly one context. When a context goes out of scope, its channels and sinks are disconnected, and logging stops.
    • Channel: A way to log related messages sharing the same schema or type. Each channel is identified by a unique topic name (e.g., /my_topic).
    • Sink: The destination for logged messages. Common sinks include:
      • MCAP file: Local storage using foxglove::McapWriter::create.
      • WebSocket server: Local streaming using foxglove::WebSocketServer::create (defaults to 127.0.0.1:8765).
      • Remote access gateway: Remote streaming via WebRTC using foxglove::RemoteAccessGateway::create.

    If no sink is configured, log messages are simply dropped.

  8. Enable Remote Access in the C++ SDK

    main

    Remote access support provides the RemoteAccessGateway class for live visualization and teleoperation via the Foxglove platform.

    Important Requirements:

    • Enabling Remote Access (FOXGLOVE_REMOTE_ACCESS=ON) produces only a shared library (foxglove_cpp_shared). No static library is produced because the LiveKit/WebRTC dependency has strict ABI requirements.
    • ABI Compatibility: Your application must match the specific compiler and runtime requirements of the platform (see the compatibility table in the documentation) to avoid linker errors or undefined behavior.
    • Header Guard: When using the C API, the FOXGLOVE_REMOTE_ACCESS define must be present. If using CMake with the shared library, this is handled automatically. Otherwise, define FOXGLOVE_REMOTE_ACCESS manually before including foxglove-c.h.
    # Build with Remote Access enabled
make build FOXGLOVE_REMOTE_ACCESS=ON

  9. How Foxglove SDK channels and servers work in the OAK example

    main

    The integration follows a three-step pattern:

    1. Channel Creation

    The SDK uses typed channels for well-known schemas so Foxglove can visualize them automatically. For non-standard schemas like IMU, a generic JSON channel is used.

    points_channel = PointCloudChannel(topic="/oak/points")
rgb_channel = RawImageChannel(topic="/oak/rgb/image")
cal_channel = CameraCalibrationChannel(topic="/oak/rgb/calibration")
tf_channel = FrameTransformsChannel(topic="/tf")
# IMU uses JSON to maintain compatibility with ROS-like structures
imu_channel = Channel(topic="/oak/imu", message_encoding="json", schema=...)

    2. Server Initialization

    Starting the server enables broadcasting all subsequent messages to connected clients:

    server = foxglove.start_server()

    3. Data Conversion

    DepthAI packets are converted to Foxglove messages in a loop:

    • Color ImgFrame $\rightarrow$ foxglove.RawImage (bgr8 encoding).
    • PointCloudData $\rightarrow$ foxglove.PointCloud (XYZ in meters, RGB/A in uint8).
    • IMU packets $\rightarrow$ JSON messages containing accelerometer and gyroscope vectors.

    All messages use converted Unix-epoch Timestamps to ensure synchronization.

    points_channel = PointCloudChannel(topic="/oak/points")
rgb_channel = RawImageChannel(topic="/oak/rgb/image")
cal_channel = CameraCalibrationChannel(topic="/oak/rgb/calibration")
tf_channel = FrameTransformsChannel(topic="/tf")
imu_channel = Channel(topic="/oak/imu", message_encoding="json", schema=...)

server = foxglove.start_server()

  10. How to update or delete 3D scene entities

    main

    To manage entities in a 3D scene, use the SceneUpdate message, which contains both entities (to add or replace) and deletions (to remove).

    Deleting Entities

    Use SceneEntityDeletion within a SceneUpdate to remove entities.

    • type: An enum SceneEntityDeletionType specifying the action.
    • id: If type is MATCHING_ID, this must match the entity's ID.
    • timestamp: Only entities published earlier than this timestamp will be deleted.

    Adding/Replacing Entities

    Include SceneEntity objects in the entities array of a SceneUpdate. If an entity has an id that already exists in the scene, it will be replaced by the new one.

  11. Use type-safe logging with message types

    main

    The SDK provides classes for well-known message types. For type-safe logging, use these in conjunction with associated channel classes. This ensures at compile time that messages logged to a channel share a common schema.

    Example: Create a SceneUpdateChannel to log SceneUpdate messages.

    Note:

    • The foxglove.schemas module is deprecated. Use foxglove.messages instead.
    • Message classes are currently immutable and do not expose getters/setters for fields.