WizMap

You can share specific embedding maps with collaborators by constructing a URL that includes the dataURL and gridURL query parameters. These parameters point to the JSON files containing your embedding data and grid information.

Example URL structure: https://poloclub.github.io/wizmap/?dataURL=<URL_TO_DATA_NDJSON>&gridURL=<URL_TO_GRID_JSON>

To run the WizMap web application locally for development, clone the repository, install the dependencies using npm, and start the development server.

git clone git@github.com:poloclub/wizmap.git
npm install
npm run dev

After running npm run dev, navigate to http://localhost:3000 in your browser.

The UMAPPointStreamData type defines the expected format for raw UMAP data arrays. The array can vary in length depending on the metadata available for each point:

• [number, number, string]: [x, y, text]
• [number, number, string, string]: [x, y, text, year]
• [number, number, string, string, number]: [x, y, text, year, group]

export type UMAPPointStreamData =
  | [number, number, string]
  | [number, number, string, string]
  | [number, number, string, string, number];

The Packer class uses a provided Svelte tooltipStore to manage tooltip visibility and content. When a user hovers over a circle, the Packer updates the store with:

• html: A formatted string containing the phrase label and its occurrence count.
• x: The horizontal center position of the tooltip.
• y: The vertical position of the tooltip.
• show: A boolean indicating if the tooltip should be visible.

Developers can subscribe to this store to render the tooltip UI anywhere in their application.

To use WizMap for dimensionality reduction and visualization, install the wizmap package along with umap-learn using pip.

!pip install --upgrade wizmap umap-learn

WizMap requires a density grid to render the background heatmap. You must compute a Kernel Density Estimation (KDE) over a regular 2D grid and export a JSON object containing the grid values, the coordinate ranges, and optional temporal grids (for time-based filtering).

# The resulting JSON structure should look like this:
grid_density_json = {
    'grid': grid_density.astype(float).round(4).tolist(), # 2D list of density values
    'xRange': [x_min, x_max],
    'yRange': [y_min, y_max],
    'sampleSize': sample_size,
    'totalPointSize': umap_df.shape[0],
    'padded': True,
    'timeGrids': time_grids # Dictionary mapping year strings to 2D density lists
}

Use UMAP to project high-dimensional embeddings into 2D space (xs, ys) for visualization. This example uses cuml for GPU-accelerated UMAP, but standard umap-learn can be used similarly. The resulting coordinates should be bundled with metadata (titles, authors, years) into a DataFrame for export.

import cuml

n_neighbors = 60
min_dist = 0.1
reducer_cuml = cuml.UMAP(
    n_neighbors=n_neighbors,
    min_dist=min_dist,
    metric='cosine',
    n_components=2,
    verbose=False,
    random_state=20220101
)

projected_emb_cuml = reducer_cuml.fit_transform(embeddings)

WizMap can consume data in NDJSON format. Each line in the file should represent a single data point, typically containing its 2D coordinates, the associated text, and optional metadata like year or title.

# Format: [x, y, text, year, title]
import ndjson

umap_data_short = []
for i in range(len(xs)):
    cur_row = [xs[i], ys[i], texts[i], times[i], labels[i]]
    umap_data_short.append(cur_row)

with open("umap.ndjson", "w") as fp:
    ndjson.dump(umap_data_short, fp)

To generate embeddings for text data (e.g., paper abstracts), use the sentence-transformers library. You can specify a device (CPU or CUDA) and encode a list of strings in batches to optimize performance.

from sentence_transformers import SentenceTransformer
import torch

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = SentenceTransformer('all-mpnet-base-v2', device=device)

abstracts = [sentence.lower() for sentence in metadata_df['abstract']]
embeddings = model.encode(abstracts, batch_size=64, show_progress_bar=True)

When initializing the Embedding view, you can provide an EmbeddingInitSetting object to control the visibility of various map elements. Use the following keys:

• showContour: boolean
• showPoint: boolean
• showGrid: boolean
• showLabel: boolean

const settings: EmbeddingInitSetting = {
  showContour: true,
  showPoint: true,
  showGrid: true,
  showLabel: true
};