DOCSEARCHbyAlgolia
Submit Docs

Embedding Atlas

repository·main·Indexed 26 days ago

https://github.com/apple/embedding-atlas

A tool for interactive visualization, cross-filtering, and searching of large embeddings and associated metadata. It supports Python (CLI, Jupyter, and Streamlit widgets) and Web environments (React, Svelte, Vanilla JS). Key features include WebGPU-powered performance for millions of points, automatic clustering, UMAP dimensionality reduction via a high-performance Rust implementation, and linked dashboards for tabular data. It also supports AI agent access via the Model Context Protocol (MCP).

Tokens
33.8K
Snippets
73
Records
267
Agent score
86%

What's inside Embedding Atlas

  1. Overview of UMAP (Rust) implementation

    main
    The umap package is a pure Rust implementation of UMAP (Uniform Manifold Approximation and Projection) and NNDescent (approximate nearest neighbor search). It is designed to be a high-performance port of the original Python libraries umap-learn and pynndescent. It includes WebAssembly bindings for use in browser environments.

  2. Overview of Embedding Atlas capabilities

    main

    Embedding Atlas is a tool for interactive visualization of large embeddings and metadata. It allows users to:

    • Explore embeddings: Visualize 2D projections of millions of points, browse automatic clusters/labels, find nearest neighbors, and cross-filter against metadata.
    • Build dashboards: Use standard charts (bar, line, bubble, count plot, eCDF) or a composable chart spec for custom charts. Supports cross-filtering between charts.
    • Drive analysis with AI agents: Use the command line tool's MCP (Model Context Protocol) server to allow agents to query schemas, run SQL, create/modify charts, and capture screenshots.
    • Work with multimodal data: Supports text, image, audio, numeric, categorical, and time columns with specialized viewers and time-aware charts.

    Note: If a dataset lacks an embedding column, the embedding view is hidden, but linked charts, full-text search, and the instances view remain functional.

  3. Overview of Embedding Atlas features

    main

    Embedding Atlas provides interactive visualizations for large embeddings and tabular metadata.

    Key Capabilities

    • Embeddings: Automatic clustering and labeling, kernel density estimation, order-independent transparency, real-time search/nearest neighbors, and WebGPU-powered performance for millions of points.
    • Tabular Data: Linked dashboards with cross-filtering (bar, line, bubble, count plot, eCDF), custom chart specs (heatmaps, average-line overlays), and multimodal support (text, image, audio, numeric, categorical, and time).
    • AI Integration: Supports AI agent access via Model Context Protocol (MCP) for querying schemas, running SQL, and creating charts.

  4. Implement a Custom Overlay

    main

    To add an overlay to the embedding view, create a class that implements constructor(target, props), update(props), and destroy(). The props passed to the constructor will contain a proxy field. You can use proxy.location(x, y) to retrieve the pixel location of a data point at coordinates (x, y). Pass this class to the customOverlay property of EmbeddingViewMosaic.

    class CustomOverlay {
  constructor(target, props) {
    // Create the tooltip component and mount it to the target element.
    // props will contain a `proxy` field, plus any custom prop you specified.
    // You can use proxy.location(x, y) to get the pixel location of a data point at (x, y).
  }
  update(props) {
    // Update the component with new props.
  }
  destroy() {
    // Destroy the component.
  }
}

// Usage:
<EmbeddingViewMosaic
  ...
  customOverlay={{
    class: CustomOverlay,
    props: { customProp: 10 } // Pass additional props to the overlay component.
  }}
/>

  8. Use the EmbeddingAtlasWidget in Python notebooks

    main

    The EmbeddingAtlasWidget allows you to embed the Embedding Atlas UI directly into notebooks (Jupyter, Marimo, Colab, VSCode, etc.) using AnyWidget.

    To display a widget with a pre-computed projection, pass the dataframe and the relevant column names for text, x-coordinates, y-coordinates, and neighbors to the constructor. You can retrieve user selections from the widget as a dataframe using the .selection() method.

    from embedding_atlas.widget import EmbeddingAtlasWidget

# Create an Embedding Atlas widget with the pre-computed projection
widget = EmbeddingAtlasWidget(df, text="description",
    x="projection_x", y="projection_y", neighbors="neighbors"
)

# Display the widget
widget

# Get selections back as a data frame
df_selection = widget.selection()

  9. Use the EmbeddingAtlas Svelte wrapper

    main

    Import EmbeddingAtlas from embedding-atlas/svelte to use the component within a Svelte application. It requires a coordinator (Mosaic coordinator) and a data object defining the table, ID, projection coordinates, and text columns.

    import { EmbeddingAtlas } from "embedding-atlas/svelte";

let coordinator: Coordinator; // The Mosaic coordinator.

<EmbeddingAtlas
  coordinator={coordinator}
  data={{
    table: "data_table",
    id: "id_column",
    projection: { x: "x_column", y: "y_column" },
    text: "text_column"
  }}
  ...
/>

  11. Implement a Custom Tooltip

    main

    To change how tooltips are displayed, provide a class to the customTooltip property. The class must implement a constructor(target, props), update(props), and destroy() method. The props passed to the constructor and update method will contain the tooltip field and any additional custom props provided.

    class CustomTooltip {
  constructor(target, props) {
    // Create the tooltip component and mount it to the target element.
    // props will contain a `tooltip` field, plus any custom prop you specified.
  }
  update(props) {
    // Update the component with new props.
  }
  destroy() {
    // Destroy the component.
  }
}

// Usage:
<EmbeddingView
  ...
  customTooltip={{
    class: CustomTooltip,
    props: { customProp: 10 } // Pass additional props to the tooltip component.
  }}
/>