UMAP (Uniform Manifold Approximation and Projection)

UMAP is a fast non-linear dimension reduction technique used for both data visualization (similar to t-SNE) and general dimension reduction. It features a scikit-learn compatible API and is designed for high performance.

Key capabilities include:

• Supervised/Semi-supervised learning: Use labels (or partial labels) to guide the dimension reduction process.
• Transforming new data: Ability to project unseen data into a previously trained embedding space.
• Speed: Significantly faster than most t-SNE implementations.

When choosing a dimension reduction method, consider the trade-off between speed and complexity. Based on performance benchmarks:

• PCA: The fastest option, but may sacrifice significant information/structure.
• UMAP: Offers a strong balance; it is slower than PCA but demonstrates significantly better scaling performance than t-SNE implementations (like MulticoreTSNE or openTSNE) as dataset size increases.
• t-SNE (MulticoreTSNE/openTSNE): Efficient for medium datasets, but UMAP typically outperforms them in scaling as you approach very large datasets (e.g., 70,000+ samples).
• MDS/Isomap/SpectralEmbedding: Generally scale poorly and become unmanageable for large datasets.

Parametric UMAP allows you to train a neural network to learn a UMAP-based transformation. This is useful for:

• Faster inference on new, unseen data.
• Robust inverse transforms.
• Autoencoder versions of UMAP.
• Semi-supervised classification.

UMAP's multi-threaded mode introduces non-deterministic behavior due to race conditions between threads during optimization.

• Default (No random_state): High performance, uses multiple CPU cores, but results are not explicitly reproducible across runs.
• Reproducible (With random_state): Lower performance, reduced multi-threading, but results are identical across runs if the same seed is used.

When re-training a ParametricUMAP model on new data, the embedding space can drift due to UMAP's invariance to rotation and translation. To prevent this and keep the space consistent, use the landmarks feature.

To implement this:

1. Select a subset of indices from your original training data to serve as landmarks.
2. Create a landmarks vector where the positions of the selected landmark points are their original embeddings, and all other positions (for the new data) are set to NaN.
3. Set the landmark_loss_weight attribute on your ParametricUMAP instance.
4. Call .fit() using the new data and the landmark_positions argument.

Audio Explorer uses UMAP to embed sound samples into a 2D space. To implement this, use MFCCs and/or WaveNet to provide an initial vector representation of the sound samples before applying UMAP to generate the 2D embedding. This allows users to group similar sounds together for rapid auditory searching.

http://doc.gold.ac.uk/~lfedd001/three/demo.html

UMAP is a stochastic algorithm. To ensure that results can be reproduced exactly, you must explicitly provide a random_state value to the umap.UMAP constructor.

Note on Performance: Setting a random_state disables certain multi-threading optimizations to ensure reproducibility. Consequently, runs with a fixed random_state will typically have a higher 'Wall time' (closer to CPU time) and will not exploit multiple CPU cores as effectively as the default stochastic runs.

UMAP supports several optional installation extras for specific functionalities:

• Plotting: Installs dependencies for plotting functionality.
• Parametric UMAP: Installs a CPU-only version of Tensorflow.
• TBB: Provides additional CPU optimizations for x86 processors.

The ESM Metagenomic Atlas uses 2D UMAP projections to group protein sequences by similarity based on a language model's internal representation. Each point represents a protein, and the position in the map reveals structural patterns at different scales. This approach allows for the visualization of massive datasets (up to 1 million points in a browser) by grouping similar sequences into clusters.

https://esmatlas.com/explore

Unlike many dimension reduction techniques that require dense input or a preliminary linear step (like TruncatedSVD), UMAP can operate directly on sparse matrix structures. This is highly efficient for large, high-dimensional datasets where most features are zero. You can pass a scipy.sparse matrix directly to the .fit() method of a UMAP instance.

import scipy.sparse
import umap

# Assuming factor_matrix is a scipy.sparse matrix
mapper = umap.UMAP(metric='cosine', random_state=42, low_memory=True).fit(factor_matrix)