viser

repository·main·Indexed 25 days ago

https://github.com/viser-project/viser

A 3D visualization library for computer vision and robotics in Python. It provides an imperative, web-based interface for visualizing 3D primitives, building interactive GUIs with elements like buttons and sliders, and managing scene interactions via a hierarchical scene tree. Key features include programmatic camera control, support for various geometry and mesh handles, and a WebSocket-based client-server architecture for efficient data streaming.

Tokens
61.7K
Snippets
87
Records
301
Agent score
82%

What's inside viser

  1. Overview of Viser features

    main

    Viser is a 3D visualization library for computer vision and robotics in Python. Key capabilities include:

    • 3D Primitives: API for visualizing 3D objects.
    • GUI Building Blocks: Interactive elements like buttons, checkboxes, text inputs, and sliders.
    • Scene Interaction: Tools for clicks, selection, and transform gizmos.
    • Camera Control: Programmatic camera control and rendering.
    • Web-based Client: Entirely web-based, making it easy to use over SSH.
  2. Understand the Dock & Panel Scope Model

    main

    Viser's layout system is organized into three hierarchical scopes. Understanding these is essential for predicting how elements behave when docked or floating:

    1. Panel: The smallest unit, representing one tab group (one or more panes with one active tab).
      • Docked: A cell within a column.
      • Floating: A cell within a window's stack.
    2. Stack: A visual column of panels.
      • Docked: A single region column.
      • Floating: The entire window's stack.
    3. Region: A screen edge (left or right) containing one or more full-height columns side-by-side.

    Relationship: Panel ⊂ Stack ⊂ Region.

  3. Understand Dock and Panel Surface Forms

    main

    The Viser dock system utilizes four distinct visual forms based on the scope and collapse state of a panel or group:

    1. Expanded Cell: The standard view for docked or floating-stacked panels. Includes a grip bar (for dragging), a tab strip (for switching panes), and a body (for content).
    2. The Bar: The collapsed rendering of a floating window. It appears as a 26px tall strip (MINIMIZED_BAR_PX) containing dimmed tab labels. A single-group window shows a + icon at the right end.
    3. The Rail: The collapsed rendering of a docked column. It appears as a 36px wide vertical strip containing spine rows (icon + rotated title) for each tab.
    4. Floating Window: A window that is not docked. It can be multi-group (with a stack handle header) or single-group (where the grip bar serves as the header).
  4. Understand the Dock & Panels design specification

    main

    The dock-ux-spec.md file contains the design decisions (D1-D58) governing the behavior of the Dock and Panel system in Viser. It defines how panels dock, collapse, resize, and interact with the layout. Key concepts include:

    • Docking Behavior: Side-docking grows the region by the newcomer's width; existing panels do not shrink (D3).
    • Collapse & Minimization: Collapse is a container-scoped state (D38). A window is a vertical stack of cells where the header persists even when collapsed (D17). Minimizing a group of stacked panels minimizes them together (D47).
    • Rails: A railed column is rendered as a fixed 36px strip (D21, D28). Rails never merge; a packed region consists of N independent strips (D44, D46).
    • Layout Model: The system uses a columns-only layout model. Regions are side-by-side full-height columns, each containing a stack of leaves (D46).
    • Mobile Support: Below the xs breakpoint, panels render as an accordion of bars in a bottom sheet (D45).
    • Resizing & Dividers: Width dividers are inert beside a railed column (D24). Height dividers support content-height snapping (D56).
  5. Configure Scene Lighting, Fog, and Environment

    main

    Enhance your 3D scene with environmental effects and lighting:

    • Lighting and Shadows: Add directional lights and ambient lighting to illuminate meshes with realistic shadows.
    • Distance Fog: Add distance-based fog to fade objects into a background color.
    • Depth Compositing: Display background images that can occlude 3D geometry based on depth values.
    • Global Up Direction: Control the global up direction for camera navigation and scene orientation.
  6. Understand Viser Dock and Panel Vocabulary

    main

    Viser uses a specific hierarchy of terms to describe the layout and docking system. Understanding these terms is essential for working with the UI structure:

    Core Units

    • Pane / Panel / Tab: A pane is the atomic unit of content (PaneId/PaneSpec). A panel is the user-facing container (represented by a PanelHandle in Python). A tab is how a pane is presented within a strip.
    • Group: An ordered set of one or more panes (tabs) with one active tab. This represents the PANEL scope and does not carry collapse state.
    • Area: A nested dockable surface located inside a panel body (a flat tab group without splits).

    Layout Containers

    • Region: A docked container on the left or right screen edge consisting of one or more columns side by side. Column widths divide the region.
    • Column: A full-height vertical stack of one or more cells. Columns carry the docked collapse flag (railed).
    • Cell (leaf): A single tab group at a specific dock position.
    • Floating window: A free-floating box containing a vertical stack of one or more groups (the STACK scope).

    Collapsed States

    • Bar: The collapsed rendering of a Floating window (a 26px handle).
    • Rail: The collapsed rendering of a Docked column (a 36px vertical strip of spine rows).
    • Rail header: The handle atop a rail; used to drag the column into a floating window or to expand it.
    • Cap: The collapsed panel handle located inside a rail; used to drag a specific group.

    Special Panels

    • Main panel: The control panel. It behaves like an ordinary group but is unmergeable and includes a titleNode header (e.g., for connection status).
    • Unmergeable panel: A panel that cannot be merged into another group. It renders a full-width header instead of a tab strip.
  7. Visualize 3D Scene Elements in Viser

    main

    Viser provides a variety of scene visualization capabilities for 3D environments. You can use the library to render the following elements:

    • Coordinate Frames: Visualize 3D coordinate systems and hierarchical transformations.
    • Point Clouds: Visualize 3D point clouds with custom colors.
    • 3D Meshes: Load and display 3D meshes from common file formats.
    • Batched Mesh Rendering: Efficiently render many instances of the same mesh using different transforms and colors.
    • Lines and Splines: Create line segments and smooth splines for wireframes and path visualization.
    • Arrows: Visualize vectors, directions, and quantities in 3D space.
    • Images: Display background images and 3D image textures.
    • Gaussian Splats: Use the built-in WebGL-based Gaussian splat renderer.
  8. Understand the Dock and Panel hierarchy

    main

    The Viser dock system is organized into three nested scopes. Understanding this hierarchy is essential for managing layouts:

    1. Panel: A single tab group.
    2. Stack: A vertical run of panels that forms a single column (e.g., a docked column or a floating window's stack).
    3. Region: A screen edge (left or right) that contains stacks.

    Additionally, a docked region is structured as a three-level tree:

    • A Region holds Columns side-by-side.
    • A Column stacks Cells top-to-bottom.
    • A Cell contains exactly one group.
  9. Understand Dock collapse and rendering states

    main

    Collapse is a single state stored on a container. When a scope collapses, it renders into a specific form based on its geometry:

    • Floating Panel collapses into a 26px bar.
    • Floating Stack collapses into a window of bars.
    • Docked Column collapses into a 36px rail.

    Note that bars and rails are simply different renderings of the same 'collapsed' state. Moving a collapsed element between different layout contexts (e.g., from floating to docked) converts the rendering but does not change the underlying state.

  10. Understand Dock & Panel Model Integrity

    main

    The Viser Dock system uses a 'correct by construction' model to prevent common layout bugs. Developers should be aware of the following structural constraints:

    • Three-level shape: The hierarchy follows DockRegion { columns } $\rightarrow$ DockColumn { children } $\rightarrow$ leaves. Vertical stacking is only permitted inside a column.
    • Identity and Types: The system uses distinct, mutually unassignable types for identifiers: PaneId, GroupId, WindowId, NodeId, and AreaId.
    • Window Height: Height is defined as either {mode: "auto"} or {mode: "pinned"; px}.
    • Float Ownership: Floating elements use an optional anchor: {x; y} object. The presence of this object indicates the element is anchored.
    • Tab Groups: TabGroup.activeId can be a PaneId or null (it does not use an empty string sentinel).
    • Placement Protocol: Layout is managed via per-axis (counter, runId) stamps. Reconnects are handled via an explicit phase that ends when the server sends a ReplayDoneMessage.
  11. Handle interaction events in Viser

    main
    Viser uses a set of specialized event classes to pass information to callback functions when user interactions occur. These events are triggered by actions such as GUI updates, scene clicks, node dragging, or pointer movements. When setting up callbacks for interactive elements, you should expect one of the following event types to be passed to your function.
  12. Animate camera transitions smoothly

    main

    You can create smooth camera animations by interpolating between the current camera pose and a target pose using SE(3) transformations.

    1. Calculate the current transformation matrix ($T_{world_current}$) from client.camera.wxyz and client.camera.position.
    2. Calculate the target transformation matrix ($T_{world_target}$) based on the target frame's pose.
    3. Compute the relative transformation: $T_{current_target} = T_{world_current}^{-1} \times T_{world_target}$.
    4. Use tf.SE3.exp(T_current_target.log() * alpha) to interpolate between the two poses over a series of steps.
    5. Apply updates within a client.atomic() block to maintain smoothness.
    import time
    import numpy as np
    import viser
    import viser.transforms as tf
    
    # ... inside a client callback ...
    # T_world_current and T_world_target are pre-calculated SE(3) matrices
    T_current_target = T_world_current.inverse() @ T_world_target
    
    for j in range(20):
        # Interpolate using SE(3) exponential map
        T_world_set = T_world_current @ tf.SE3.exp(
            T_current_target.log() * j / 19.0
        )
    
        with client.atomic():
            client.camera.wxyz = T_world_set.rotation().wxyz
            client.camera.position = T_world_set.translation()
    
        client.flush()
        time.sleep(1.0 / 60.0)
    
    # Set look_at to the target position so mouse orbit works correctly
    client.camera.look_at = target_position