pyribs: Rapid Illumination of Behavior Space

repository·master·Indexed 42 days ago

A modular Python library for Quality Diversity (QD) optimization implementing the RIBS framework. It allows users to compose QD algorithms by interchanging archives, emitters, and schedulers using an ask-tell interface. Supported algorithms include MAP-Elites, CMA-ME, BOP-Elites, and Novelty Search, with built-in support for GridArchive, CVTArchive, and the Continuous QD (CQD) Score.

Tokens: 17.7K
Snippets: 48
Records: 69
Agent score: 54%

What's inside pyribs

New algorithms supported in pyribs
master
The following algorithms have been added in recent versions:
- Discount Model Search (DMS): Uses ribs.archives.DiscountArchive and ribs.discount_models.DiscountModelManager.
- Dominated Novelty Search (DNS): Uses ribs.archives.DNSArchive.
- Novelty Search with Local Competition (NSLC): Uses ribs.emitters.rankers.NSLCRanker and ribs.emitters.rankers.NSLCClassicRanker.
- Density Descent Search with Continuous Normalizing Flows (DDS-CNF): Uses ribs.archives.DensityArchive.
- Novelty Search with Threshold Decay: Supported in ProximityArchive via automatic decay of novelty_threshold.
https://github.com/icaros-usc/pyribs/blob/master/docs/whats-new.md
How the RIBS framework works
master
Pyribs implements the Rapid Illumination of Behavior Space (RIBS) framework, which decomposes Quality Diversity (QD) optimization into three modular components:
1. Archive: Saves generated solutions within a defined measure space (often represented as a heatmap/grid).
2. Emitters: Algorithms that generate new candidate solutions and respond to feedback about how those solutions were evaluated and inserted into the archive.
3. Scheduler: The central controller that manages the interaction between the archive and emitters. It provides the interface for the user to request solutions and report evaluations.
The Optimization Loop (Ask-Tell Interface)
Pyribs follows an ask-tell pattern:
1. Ask: The user calls scheduler.ask() to request new candidate solutions from the emitters.
2. Evaluate: The user evaluates these solutions to obtain objective values (what is being optimized) and measure values (the behavioral characteristics/coordinates in measure space).
3. Tell: The user calls scheduler.tell(objectives, measures) to pass the results back to the scheduler. The scheduler updates the archive and provides feedback to the emitters so they can update their internal state.
https://github.com/icaros-usc/pyribs/blob/master/README.md
Understand pyribs v0.5.0 terminology changes
master
Version 0.5.0 introduced several breaking changes to terminology to align with recent literature. If you are migrating from v0.4.0, note the following mappings:
- measures (formerly behaviors or behavior_values): Refers to the properties of solutions that QD optimization varies.
- cells (formerly bins): The units used to partition the archive.
- schedulers (formerly optimizers): The components in the ribs.schedulers module (e.g., Scheduler, BanditScheduler) that manage the QD process.
- Batch arguments: Many methods now support batch operations. Individual arguments are named solution, objective, measures, and metadata, while their batch counterparts are solution_batch, objective_batch, measures_batch, and metadata_batch.
https://github.com/icaros-usc/pyribs/blob/master/docs/whats-new/v0.5.0.md
Integrate pyribs with Optuna via OptunaHub plugins
master
You can use pyribs algorithms within the Optuna hyperparameter optimization framework by using specialized plugins available on OptunaHub.
There are two primary ways to integrate these tools:
1. Using CMA-MAE as an Optuna Sampler: The CMA-MAE Sampler plugin allows you to use the CMA-MAE algorithm (implemented in pyribs) as a sampler within an Optuna study. This is useful for searching for diverse, high-performing hyperparameters.
2. Visualizing Optuna results with pyribs tools: The Pyribs Visualization Wrappers plugin provides wrappers around pyribs visualization tools, allowing you to visualize the results generated by the CMA-MAE Sampler.
https://github.com/icaros-usc/pyribs/blob/master/tutorials/optuna.md
Install dev dependencies for running benchmarks
master
To run the benchmark scripts located in the benchmarks/ directory, you must install the development dependencies for the ribs package. You can do this using pip with the [dev] extra.
If you are on macOS, you may need to wrap the installation target in quotations to prevent shell errors.
```
# Standard installation
pip install ribs[dev]

# Editable installation (recommended for development)
pip install -e .[dev]

# macOS specific (if shell errors occur)
pip install "ribs[dev]"
```
https://github.com/icaros-usc/pyribs/blob/master/benchmarks/README.md
Explore pyribs examples
master
The pyribs repository provides several standalone Python examples for testing algorithms and metrics. Each example is designed to be run as a single file. To use them, you must first install the specific dependencies listed at the top of each file, then execute the script.
Available examples include:
- Sphere Benchmark: Demonstrates setting up recent Quality Diversity (QD) algorithms applied to the sphere linear projection benchmark.
- Lunar Lander (Distributed): An extended version of the Lunar Lander tutorial that uses Dask for distributed evaluations and includes a command-line interface.
- BOP-Elites: Demonstrates running Bayesian Optimization of Elites, which uses Bayesian optimization to model objective and measure functions for solution selection.
- CQD Score: Demonstrates how to compute the Continuous QD Score, a metric for evaluating the tradeoff between objective values and distance to desired measure values.
https://github.com/icaros-usc/pyribs/blob/master/docs/examples.md
Implement Novelty Search with ProximityArchive
master
To implement Novelty Search, use the ribs.archives.ProximityArchive class. You can visualize the results of a proximity archive using the ribs.visualize.proximity_archive_plot function. For a detailed walkthrough, refer to the ns_maze tutorial.
https://github.com/icaros-usc/pyribs/blob/master/docs/whats-new/v0.8.0.md

Visualize archive distributions with histograms and ECDFs

master

Use ribs.visualize to plot the distribution of objective values in an archive.

archive_histogram: Plots a histogram of objective values for a single archive. Supports a cmap argument for coloring.
archive_ecdf: Plots the empirical cumulative distribution function (ECDF) for a single archive. Use complementary=True to plot the ECCDF.
aggregate_cdf: Plots the CDF or CCDF aggregated over multiple archives. This is useful for comparing performance across multiple runs. Supports estimator (e.g., 'median') and errorbar (e.g., 'iqr') arguments.

import numpy as np
import matplotlib.pyplot as plt
from ribs.archives import GridArchive
from ribs.visualize import archive_histogram, archive_ecdf

# Setup archive
archive = GridArchive(solution_dim=2, dims=[100, 100], ranges=[(-1, 1), (-1, 1)])
x = np.random.uniform(-1, 1, 10000)
y = np.random.uniform(-1, 1, 10000)
archive.add(solution=np.stack((x, y), axis=1), objective=-(x**2 + y**2), measures=np.stack((x, y), axis=1))

# Plotting
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16,6))
archive_histogram(archive, ax=ax1, cmap="magma")
ax1.set(title="Histogram", xlabel="Objective", ylabel="Num. Elites")
archive_ecdf(archive, ax=ax2, complementary=True, stat="count", color="#7e57c2")
ax2.set(title="ECCDF", xlabel="Objective", ylabel="Num. Elites")
plt.show()

Implement Density Descent Search with DensityArchive
master
Density Descent Search is implemented via the ribs.archives.DensityArchive class. A practical example of its usage can be found in the sphere example documentation.
https://github.com/icaros-usc/pyribs/blob/master/docs/whats-new/v0.8.0.md
Run pyribs tutorials locally
master
To run the pyribs tutorials on your own machine, follow these steps:
1. Download the notebooks: Get the .ipynb files from the pyribs GitHub repository.
2. Install Jupyter Lab: Use pip to install the Jupyter environment.
3. Launch Jupyter Lab: Start the server to open the interface in your browser.
4. Execute notebooks: Open the downloaded notebooks. Note that the notebooks use cell magic to automatically install any required dependencies during execution.
```
# Install Jupyter Lab
pip install jupyterlab

# Start Jupyter Lab
jupyter lab
```
https://github.com/icaros-usc/pyribs/blob/master/docs/tutorials.md
Use pycma in Emitters
master
You can use the pycma library within EvolutionStrategyEmitter by setting the es parameter to "pycma_es".
Requirement: You must have pycma installed. You can install it via pip install cma or by using the pyribs extra: pip install "ribs[pycma]".
```
from ribs.emitters import EvolutionStrategyEmitter

emitter = EvolutionStrategyEmitter(
    archive,
    x0=[0.0] * 10,
    sigma0=0.1,
    es="pycma_es"
)
```
https://github.com/icaros-usc/pyribs/blob/master/docs/whats-new/v0.7.0.md

Install pyribs

master

Pyribs is distributed in three ways depending on your needs. If you do not require visualization, use the Default distribution. If you need visualization (currently via Matplotlib), use the Visualize distribution. The All distribution installs everything.

Name	PyPI Package	Conda Package	Description
Default	`ribs`	`pyribs`	Basic pyribs package
Visualize	`ribs[visualize]`	`pyribs-visualize`	Adds visualization dependencies
All	`ribs[all]`	`pyribs-all`	Installs all extra distributions

Note: On macOS, you may need to use quotations when installing the visualize extra via pip.

pip install "ribs[visualize]"
# or
conda install pyribs-visualize

pyribs: Rapid Illumination of Behavior Space

What's inside pyribs

The Optimization Loop (Ask-Tell Interface)