DOCSEARCHbyAlgolia
Submit Docs

BoTorch Documentation

website·Indexed 35 days ago

https://botorch.org/

A library for Bayesian Optimization (BO) providing tools for model fitting, acquisition function optimization, and advanced BO strategies. Documentation covers versions v0.16.0, v0.16.1, and v0.17.0, featuring tutorials on TuRBO-1, SAASBO, BAxUS, and various Gaussian Process implementations including Warped and High Order GPs.

Tokens
686.4K
Snippets
2K
Records
3K
Agent score
52%

What's inside BoTorch

  1. Overview of Composite Bayesian Optimization with HOGP

    v0.17.0
    Composite Bayesian Optimization is used when an unknown black-box function $f$ maps input parameters to several outputs, and a known function $g$ describes the quality of those outputs. The goal is to find input parameters $x$ that maximize the composite objective $g(f(x))$. In BoTorch, the High Order Gaussian Process (HOGP) is used as the surrogate model for $f$ because it is specifically designed to operate over tensors or multi-dimensional arrays, exploiting their structure for efficiency.

  2. Overview of the qKnowledgeGradient acquisition function

    v0.15.0
    The Knowledge Gradient (KG) is a look-ahead acquisition function that quantifies the expected increase in the maximum of a modeled black-box function after obtaining additional observations. BoTorch implements a generalized variant called qKnowledgeGradient. Unlike traditional KG, which is computationally expensive and requires solving inner optimization problems for every gradient step, BoTorch's "one-shot" formulation treats the optimization as a deterministic problem by jointly optimizing the candidate points and the fantasy points.

  3. Overview of Information-Theoretic (IT) Acquisition Functions

    v0.17.2
    Information-theoretic acquisition functions quantify the utility of an input $\mathbf{x}$ based on the mutual information (MI) between the potential observation $\mathbf{y}$ and a specific statistic of the function $S(f)$. The general formula is: $\alpha^{\text{IT}}(\mathbf{x}|D_n) = \text{MI}(\mathbf{y}; S(f)| \mathbf{x}, D_n) = H[p(\mathbf{y}|D_n)] - \mathbb{E}_{p(S(f)|D_n)}[H[p(\mathbf{y}| \mathbf{x}, D_n, S(f))]]$, where $D_n$ is the dataset of sampled inputs and observations, and $H$ denotes differential entropy.

  4. Overview of BoTorch Objectives

    v0.14.0

    In BoTorch, an objective is a module used to transform model outputs into a scalar function for optimization. Common use cases include scalarizing outputs for multi-output models or implementing optimization subject to outcome constraints (e.g., weighting by probability of feasibility).

    While analytic acquisition functions require affine transformations to maintain Gaussian assumptions, MC-based acquisition functions allow for general transformations as long as gradients can be back-propagated. All BoTorch objectives derive from MCAcquisitionObjective. Built-in options include LinearMCObjective (for linear combinations) and ConstrainedMCObjective (using sigmoid approximations for constraints).

  5. Overview of BoTorch Objectives

    v0.17.0

    In BoTorch, an objective is a module used to transform model outputs into a scalar function for optimization. Common use cases include scalarizing outputs for multi-output models or implementing optimization subject to outcome constraints (e.g., weighting by probability of feasibility).

    While analytic acquisition functions require affine transformations to maintain Gaussian assumptions, MC-based acquisition functions allow for general transformations as long as gradients can be back-propagated. All BoTorch objectives derive from MCAcquisitionObjective. Built-in examples include LinearMCObjective (for linear combinations) and ConstrainedMCObjective (using sigmoid approximations for constraints).

  6. Overview of Max-Value Entropy Search (MES) in BoTorch

    v0.14.0
    Max-value entropy search (MES) is an acquisition function used in Bayesian Optimization to quantify the information gain about the maximum of a black-box function by observing candidate sets. BoTorch provides implementations of the MES acquisition function, including a multi-fidelity (MF) version with support for trace observations.

  7. Overview of the GIBBON acquisition function

    v0.15.0

    GIBBON (General-purpose Information-Based Bayesian OptimisatioN) is an information-theoretic acquisition function designed for Bayesian Optimization. It serves as a computationally efficient, closed-form lower bound to the batch Max-value Entropy Search (MES) function. Unlike batch MES, which often requires expensive numerical integrations and 'fantasy' observations, GIBBON avoids this overhead, making it suitable for tasks where objective function query costs are not large enough to justify high optimization overheads.

    Mathematically, GIBBON balances two goals:

    1. Diversity: It encourages diversity within a batch by penalizing predictive correlation between points (via the determinant of the correlation matrix).
    2. Information Gain: It targets areas of the search space that provide the most information about the maximum value of the objective function.

  8. Overview of Max-value Entropy Search (MES) in BoTorch

    Max-value Entropy Search (MES) is an acquisition function used in Bayesian Optimization to quantify the information gain about the maximum of a black-box function by observing it at a candidate set of points. BoTorch provides implementations of the standard MES acquisition function as well as a multi-fidelity (MF) version that supports trace observations.

  9. Overview of the GIBBON acquisition function

    v0.13.0

    GIBBON (General-purpose Information-Based Bayesian OptimisatioN) is an information-theoretic acquisition function designed for Bayesian Optimization (BO) where objective function query costs are not high enough to justify the significant computational overhead of exact batch Max-value Entropy Search (MES).

    It functions as a closed-form lower bound on the batch MES function. The acquisition value is decomposed into two terms: one that encourages diversity within the batch (based on the determinant of the correlation matrix between batch elements) and another that targets areas providing high information about the objective function's maximum value.

  10. Overview of OrthogonalAdditiveGP

    The OrthogonalAdditiveGP is a Gaussian process model that decomposes predictions into interpretable per-dimension components. It is based on the Orthogonal Additive Kernel, which orthogonalizes base kernels via Gauss-Legendre quadrature. This approach is particularly useful for functions where the output is roughly a sum of independent effects from each input dimension: $f(x) = c + \sum_{i=1}^d f_i(x_i)$.

    Key advantages include:

    1. Sample Efficiency: By learning $d$ independent 1D functions instead of one $d$-dimensional function, it requires significantly fewer training points to achieve accuracy in high-dimensional spaces.
    2. Interpretability: It allows for the extraction of posterior distributions for each individual component $f_i$, revealing the specific contribution of each input dimension.

  11. Overview of Sparse Axis-Aligned Subspace Bayesian Optimization (SAASBO)

    SAASBO is designed for high-dimensional Bayesian optimization (effective for problems with hundreds of dimensions) where the evaluation budget is limited. It prevents overfitting in high-dimensional spaces by placing strong hierarchical sparsity priors (Half-Cauchy distributions) on the inverse lengthscales. Inference is performed using Hamiltonian Monte Carlo (HMC) via the No-U-Turn-Sampler (NUTS). Note that the computational overhead of SAASBO scales cubically with the number of datapoints, making it less feasible for budgets exceeding a few hundred evaluations.

  12. Overview of OrthogonalAdditiveGP

    The OrthogonalAdditiveGP is a Gaussian process model that assumes the target function is approximately additive, meaning the output is a sum of independent effects from each input dimension: $f(x) = c + \sum_{i=1}^d f_i(x_i)$. It uses an Orthogonal Additive Kernel that orthogonalizes base kernels via Gauss-Legendre quadrature.

    Key advantages include:

    1. Sample Efficiency: It learns $d$ independent 1D functions instead of one $d$-dimensional function, requiring significantly fewer training points to achieve low error in high-dimensional spaces.
    2. Interpretability: It allows for the extraction of posterior distributions for each individual component $f_i$, revealing the specific contribution of each input dimension to the final output.