MapillaryJS

MapillaryJS is a client-side JavaScript library designed for building interactive street imagery map experiences. It allows developers to display street-level imagery in a web browser or client and provides tools to augment that experience with custom data.

Core Capabilities

• Interactivity: Smooth navigation (panning, zooming) for both street-level and map cameras.
• Navigation Graph Construction: Uses S2 cell-based graph creation.
• Image Tiling: On-demand, full-resolution image rendering.
• Undistortion: Ensures textures and camera frames are undistorted within a virtual 3D world.
• Spatial Rendering: Supports rendering of point clouds, camera frames, and GPS positions.

To provide MapillaryJS with custom data, follow these three core requirements:

1. Extend DataProviderBase: Implement the required abstract methods defined in the base class.
2. Map to Ent Format: Ensure your custom data is converted into the MapillaryJS 'ent' (entity) format.
3. Attach to Viewer: Pass your provider instance to the Viewer via the ViewerOptions.dataProvider option.

For implementations that fetch files from a server, the OpenSfM data provider is a recommended reference.

MapillaryJS is designed to be extensible through several core APIs that allow you to augment or override its default behavior. Depending on your goal, you can use one of the following extension points:

• Data Provider API: Use DataProviderBase to provide your own data sources.
• Custom Render API: Implement the ICustomRenderer interface to render custom 3D models, meshes, or animations using WebGL or Three.js.
• Custom Camera Control API: Implement ICustomCameraControls to add custom interactivity or camera movement logic.

Custom rendered objects in MapillaryJS use geodetic coordinates. To place them correctly in the local 3D scene, you must convert their geodetic position (lat, lng, alt) to local topocentric (ENU - East, North, Up) coordinates relative to the current MapillaryJS reference coordinate. Use the geodeticToEnu helper function for this conversion.

function geoToPosition(geoPosition, reference) {
  return geodeticToEnu(
    geoPosition.lng,
    geoPosition.lat,
    geoPosition.alt,
    reference.lng,
    reference.lat,
    reference.alt,
  );
}

MapillaryJS uses different 2D coordinate systems for UI and image processing:

Container Pixel Coordinates

These are coordinates relative to the viewer container.

• Origin: Top-left corner (0,0).
• Axes: X increases to the right, Y increases downwards.
• Units: Pixels.

Basic Image Coordinates

These represent points in the original image, adjusted for orientation. They are useful for mapping points to the actual image content regardless of zoom or pan.

• Origin: Top-left corner (0,0).
• Axes: X increases to the right, Y increases downwards.
• Range: Normalized from 0 to 1 on both axes.

Note: You can convert between Container Pixel Coordinates and Basic Image Coordinates for the current image. Because the image can be panned and zoomed independently of the container, the conversion results will change based on the current viewing direction.

To successfully integrate 3D objects into MapillaryJS using WebGL:

1. Implement the ICustomRenderer interface: This defines the lifecycle methods required by the viewer.
2. Define object positioning: Ensure your objects have a geoPosition (latitude, longitude, and altitude) or a position relative to a geo reference.
3. Coordinate Translation: Use the MapillaryJS geo reference parameter to translate your objects from geographic coordinates to local topocentric coordinates.
4. Registration: Use viewer.addCustomRenderer() to inject your renderer into the active Viewer instance.

MapillaryJS uses a right-handed local topocentric (ENU) coordinate system, whereas Three.js uses a different default system. If you render Three.js objects directly in MapillaryJS without transformation, they will appear with incorrect positions and rotations.

To align Three.js coordinates with MapillaryJS, you must rotate the coordinates 90 degrees counter-clockwise around the X-axis. Conversely, to transform MapillaryJS coordinates to Three.js, apply a 90-degree clockwise rotation around the X-axis.

MapillaryJS utilizes two primary 3D coordinate systems depending on the context of your work:

1. Geodetic (WGS84): Used for general interaction with the MapillaryJS API. It consists of longitude (degrees), latitude (degrees), and altitude (meters).
2. Local Topocentric (ENU): Also known as the World reference frame. It is used for tasks like writing custom renderers. It uses an East, North, Up axis system where all values are in meters.

Coordinate Mapping

You can create a bidirectional link between the MapillaryJS Viewer and a Mapbox map.

• Viewer to Map: Use viewer.getposition() and viewer.getfieldofview() to update the Mapbox map position as the user navigates the imagery.
• Map to Viewer: Trigger viewer navigation based on interactions (like clicking or moving) on the Mapbox map.
• Marker Synchronization: Use the Marker component to create and edit markers in both the viewer and Mapbox, keeping them synchronized.