DOCSEARCHbyAlgolia
Submit Docs

streetscape.gl

repository·master·Indexed 21 days ago

https://github.com/aurora-opensource/streetscape.gl

A visualization engine and toolkit for 3D maps and autonomous robotics data using the XVIZ protocol. Built with React and WebGL-powered frameworks from vis.gl, it provides high-performance rendering, 3D context navigation, log playback, and support for declarative UI. The toolkit includes @streetscape.gl/core for fundamental loaders and @streetscape.gl/layers for specialized deck.gl map layers, such as ImageryLayer for ground imagery and LaneLayer for lane markers. It also provides a streetscape_avs module for interacting with XVIZ data within Jupyter environments.

Tokens
65.1K
Snippets
190
Records
281
Agent score
76%

What's inside streetscape.gl

  1. What is the Autonomous Visualization System (AVS)?

    master

    AVS is a standardized framework for describing and visualizing autonomous vehicle data, specifically perception, motion, and planning data. It is composed of two primary pillars that work together:

    1. XVIZ: The data specification and management layer. It provides a formal, stream-oriented way to describe scenes changing over time using primitives (e.g., LiDAR point clouds, camera images, trajectories).
    2. streetscape.gl: The component toolkit for building web applications. It consumes XVIZ data to provide 3D viewports, charts, tables, and video players.

    Developers use AVS to avoid building custom visualization software from scratch, allowing them to focus on core autonomy capabilities like drive systems, mapping, and simulation.

  4. Overview of streetscape.gl

    master

    streetscape.gl is a visualization toolkit designed for autonomy and robotics data encoded using the XVIZ protocol. It provides a collection of composable React components that enable users to visualize and interact with XVIZ data in a 3D environment.

    Key capabilities include:

    • Data Loading: Load XVIZ data from static files or via a stream server.
    • 3D Visualization: Interactive 3D context with base map overlays, viewport navigation (pan, rotate, zoom), object selection for annotations, and primitive stream toggling.
    • Camera Control: Customizable camera modes such as top-down, perspective, and driver views.
    • Playback: Playback of logs of unlimited length with forward and backward seeking.
    • Declarative UI: Support for rendering XVIZ declarative UI, allowing backend or on-robot systems to define interactive debugging components like tables, plots, and videos.
    • Performance: Optimized for high performance and memory efficiency with highly stylable components.

  7. Use an external state manager with streetscape.gl

    master

    By default, streetscape.gl React components like LogViewer and PlaybackControl manage their own internal state (e.g., camera view state or playback timestamp). For advanced use cases, you can take control of this state by passing state values as props and using callback props to update your external state manager (like React state, Redux, or a parent component).

    This allows you to:

    • Synchronize streetscape.gl components with non-streetscape.gl React components.
    • Intercept or override user interactions.
    • Programmatically control the camera, object styling, or playback timeline.

  9. How streetscape.gl works

    master

    streetscape.gl is a React-based toolkit built on top of WebGL for visualizing XVIZ-compliant data in web applications. It is designed for high performance and composability.

    Core Capabilities:

    • Data Consumption: Specifically designed to consume data in the XVIZ protocol.
    • Component Library: Provides drop-in-ready components for:
      • 3D viewports
      • Charts
      • Tables
      • Videos
    • Automated Synchronization: Handles complex visualization challenges such as time synchronization across different streams, coordinate system transformations, camera management, and dynamic styling.
    • Performance: Optimized for real-time playback and smooth interaction with scenes containing hundreds of thousands of geometries.
    • Extensibility: Components are highly styleable and extensible, allowing developers to tailor the UI to specific workflows like triaging, labeling, debugging, or remote assistance.

  10. Understand the XVIZLoaderInterface

    master

    The XVIZLoaderInterface is the base class responsible for handling the loading and synchronization of an XVIZ log. It provides a unified interface for different loading strategies, including:

    • XVIZStreamLoader: For streaming data.
    • XVIZLiveLoader: For live data.
    • XVIZFileLoader: For loading from files.

    Users interact with the loader via lifecycle actions (connect/close), event listeners (ready/update/finish/error), and methods to control the playback state (seek/lookAhead).

  11. How XVIZ data works

    master

    XVIZ is a stream-oriented data specification designed for autonomous systems. It allows for a declarative, structured view of a scene that can be navigated like a video (random seeking) or inspected like an HTML document.

    Key Concepts:

    • Streams: A series of discrete updates occurring at specific times using specific primitive types.
    • Primitives: Objects representing specific data types, such as:
      • LiDAR point clouds
      • Camera images
      • Object bounds
      • Trajectories
      • Vehicle speed over time
      • Predicted plans
    • Styling: Primitives can be individually styled or assigned a style class at the stream level.
    • Organization: Streams are organized via hierarchical naming. A metadata section defines streams, types, relative transforms, declarative UI panels, and style classes.

  12. Use LogViewer as a stateless component

    master

    By default, LogViewer is stateful and manages camera and object states internally. To use it as a stateless component (e.g., when integrating with a state manager like Redux), you must manually provide the following props:

    To control camera/view state:

    • viewState: Object containing longitude, latitude, zoom, pitch, and bearing.
    • viewOffset: Object containing x (horizontal offset in pixels), y (vertical offset in pixels), and bearing (camera rotation relative to car heading).

    To control object selection state:

    • objectStates: An object mapping state names to object IDs and their selection status:
      {
  [state_name_1]: {
    [object_id_1]: true,
    [object_id_2]: false
  }
}

    To handle changes:

    • onViewStateChange: Callback receiving newState (containing viewState and viewOffset).
    • onObjectStateChange: Callback receiving the new objectStates object.