DOCSEARCHbyAlgolia
Submit Docs

BoTorch Documentation

repository·main·Indexed 25 days ago

https://github.com/meta-pytorch/botorch

A modular and extensible library for Bayesian Optimization built on PyTorch. BoTorch provides primitives for probabilistic models, acquisition functions, and optimizers, leveraging PyTorch's auto-differentiation and GPU support. It includes utilities for Gaussian Process models, such as SingleTaskGP, and tools for efficient Leave-One-Out cross-validation.

Tokens
15.4K
Snippets
33
Records
63
Agent score
85%

What's inside BoTorch

  1. Explore BoTorch tutorial categories

    main

    BoTorch tutorials are organized into three main categories to help you learn different levels of control:

    1. Full Optimization Loops: Guides for implementing end-to-end Bayesian optimization loops, useful for non-standard settings or when you need granular control over the BO loop.
    2. Bite-Sized Tutorials: Focused on specific tasks encountered when customizing BO algorithms, such as:
      • Writing a custom acquisition function.
      • Using a custom zero-th order optimizer (e.g., optimizing with CMA-ES).
    3. Advanced Usage: Showcases complex applications, such as performing BO in the latent space of a variational auto-encoder (VAE) for image-based objective functions.

  2. Quickstart: Bayesian Optimization loop in BoTorch

    main

    A standard Bayesian optimization loop involves three main steps:

    1. Fit a Gaussian Process model to data: Use SingleTaskGP and fit_gpytorch_mll. Note: Double precision (torch.double) is highly recommended for GPs.
    2. Construct an acquisition function: e.g., LogExpectedImprovement.
    3. Optimize the acquisition function: Use optimize_acqf to find the next candidate.
    import torch
from botorch.models import SingleTaskGP
from botorch.models.transforms import Normalize
from botorch.fit import fit_gpytorch_mll
from gpytorch.mlls import ExactMarginalLogLikelihood
from botorch.acquisition import LogExpectedImprovement
from botorch.optim import optimize_acqf

# 1. Fit a Gaussian Process model to data
train_X = torch.rand(10, 2, dtype=torch.double) * 2
Y = 1 - (train_X - 0.5).norm(dim=-1, keepdim=True)
Y += 0.1 * torch.rand_like(Y)

gp = SingleTaskGP(
    train_X=train_X,
    train_Y=Y,
    input_transform=Normalize(d=2),
)
mll = ExactMarginalLogLikelihood(gp.likelihood, gp)
fit_gpytorch_mll(mll)

# 2. Construct an acquisition function
logEI = LogExpectedImprovement(model=gp, best_f=Y.max())

# 3. Optimize the acquisition function
bounds = torch.stack([torch.zeros(2), torch.ones(2)]).to(torch.double)
candidate, acq_value = optimize_acqf(
    logEI, bounds=bounds, q=1, num_restarts=5, raw_samples=20,
)
    import torch
from botorch.models import SingleTaskGP
from botorch.models.transforms import Normalize
from botorch.fit import fit_gpytorch_mll
from gpytorch.mlls import ExactMarginalLogLikelihood
from botorch.acquisition import LogExpectedImprovement
from botorch.optim import optimize_acqf

# 1. Fit a Gaussian Process model to data
train_X = torch.rand(10, 2, dtype=torch.double) * 2
Y = 1 - (train_X - 0.5).norm(dim=-1, keepdim=True)
Y += 0.1 * torch.rand_like(Y)

gp = SingleTaskGP(
    train_X=train_X,
    train_Y=Y,
    input_transform=Normalize(d=2),
)
mll = ExactMarginalLogLikelihood(gp.likelihood, gp)
fit_gpytorch_mll(mll)

# 2. Construct an acquisition function
logEI = LogExpectedImprovement(model=gp, best_f=Y.max())

# 3. Optimize the acquisition function
bounds = torch.stack([torch.zeros(2), torch.ones(2)]).to(torch.double)
candidate, acq_value = optimize_acqf(
    logEI, bounds=bounds, q=1, num_restarts=5, raw_samples=20,
)

  3. Build the BoTorch API reference HTML

    main
    To generate the HTML documentation, navigate to the botorch/sphinx directory and run the make html command. The generated output will be located in the botorch/sphinx/build directory. The entry point for the documentation is botorch/sphinx/build/html/index.html.

  6. Implement a basic Bayesian Optimization loop

    main

    A standard Bayesian Optimization loop in BoTorch consists of three main steps: fitting a Gaussian Process model, constructing an acquisition function, and optimizing that acquisition function to find the next candidate point.

    import torch
from botorch.models import SingleTaskGP
from botorch.models.transforms import Normalize
from botorch.fit import fit_gpytorch_mll
from gpytorch.mlls import ExactMarginalLogLikelihood
from botorch.acquisition import LogExpectedImprovement
from botorch.optim import optimize_acqf

# 1. Fit a Gaussian Process model to data
train_X = torch.rand(10, 2, dtype=torch.double) * 2
# explicit output dimension -- Y is 10 x 1
train_Y = 1 - (train_X - 0.5).norm(dim=-1, keepdim=True)
train_Y += 0.1 * torch.rand_like(train_Y)

gp = SingleTaskGP(
    train_X=train_X,
    train_Y=train_Y,
    input_transform=Normalize(d=2),
)
mll = ExactMarginalLogLikelihood(gp.likelihood, gp)
fit_gpytorch_mll(mll)

# 2. Construct an acquisition function
logNEI = LogExpectedImprovement(model=gp, best_f=train_Y.max())

# 3. Optimize the acquisition function
bounds = torch.stack([torch.zeros(2), torch.ones(2)]).to(torch.double)
candidate, acq_value = optimize_acqf(
    logNEI, bounds=bounds, q=1, num_restarts=5, raw_samples=20,
)

  7. Perform an editable/development installation of BoTorch

    main

    If you are contributing to BoTorch, install it in editable mode. It is recommended to install the development versions of gpytorch and linear_operator first.

    Bare-bones editable install:

    git clone https://github.com/meta-pytorch/botorch.git
cd botorch
pip install -e .

    Editable install with development and tutorial dependencies:

    git clone https://github.com/meta-pytorch/botorch.git
cd botorch
pip install -e ".[dev, tutorials]"

    Note: dev includes tools for testing, linting, and docs building. tutorials installs packages required to run tutorial notebooks.

  8. Build and serve Sphinx API documentation locally

    main

    Sphinx generates the API reference from source docstrings. To build and view it locally:

    1. Navigate to the sphinx/ directory.
    2. Run make html.
    3. The output is located in sphinx/build/html/.
    4. To view the docs, start a local Python server from the build directory.
    cd sphinx/
make html

# To serve the output:
cd sphinx/build/html/
python3 -m http.server 8000

  9. Run the BoTorch Docusaurus website locally

    main

    To run the Docusaurus-based website locally, you need Node.js >= 18.x and Yarn.

    1. Navigate to the website directory.
    2. Install dependencies with yarn install.
    3. Start the development server with yarn start.
    4. Access the site at http://localhost:3000.
    cd website
yarn install
yarn start

  10. Apply Parameter Constraints to candidates

    main

    Parameter constraints restrict the values of generated candidates within the input space. These are used to ensure candidates stay within specific linear (in)equality boundaries, beyond the standard bounding box defined by bounds.

    Key details:

    • Use the inequality_constraints and equality_constraints arguments in optimize_acqf (or its derivatives).
    • These constraints do not change the underlying model; they only restrict the search space for new candidate generation.
    • You can generate constrained candidates even if your existing model was trained on data points that do not satisfy these constraints.

  12. Build the BoTorch website (all-in-one)

    main

    A single shell script is provided to convert tutorials and build the website in one command. This must be executed from the repository root.

    Options:

    • -t: Execute the tutorials.
    • -b: Generate a static build.
    • -h: Show all options.
    ./scripts/build_docs.sh