GPyTorch Documentation

The gpytorch.metrics module provides functions to evaluate the performance of Gaussian Process models. Available metrics include:

• mean_absolute_error: Calculates the Mean Absolute Error (MAE).
• mean_squared_error: Calculates the Mean Squared Error (MSE).
• mean_standardized_log_loss: Calculates the Mean Standardized Log Loss.
• negative_log_predictive_density: Calculates the Negative Log Predictive Density (NLPD).
• quantile_coverage_error: Calculates the quantile coverage error.

For detailed usage instructions, refer to the metrics tutorial in the examples directory.

Likelihoods in GPyTorch define the relationship between the Gaussian Process (GP) latent function and the observed data. They are used to model the noise and the distribution of the observations.

There are two main categories of likelihoods:

1. One-Dimensional Likelihoods: Used for GPs that are distributions of scalar functions ($f(\mathbf x) \in \mathbb{R}$). To improve performance, these should extend gpytorch.likelihoods._OneDimensionalLikelihood, which uses 1D Gauss-Hermite quadrature for variance reduction instead of Monte Carlo integration.
2. Multi-Dimensional Likelihoods: Used for GPs that are distributions of vector-valued functions ($f(\mathbf x) \in \mathbb{R}^t$).

When exact computations are too costly, GPyTorch offers several approximate kernel methods to reduce asymptotic complexity:

• Sparse Gaussian Process Regression (SGPR): Approximates kernels using a set of inducing points. This is a general-purpose approximation.
• Structured Kernel Interpolation (SKI/KISS-GP): Interpolates inducing points on a regularly spaced grid. Best suited for low-dimensional data and stationary kernels.
• Structured Kernel Interpolation for Products (SKIP): An extension of SKI designed for higher-dimensional data.

Implementation examples are available in:

• SGPR_Regression_CUDA.ipynb
• KISSGP_Regression.ipynb
• Scalable_Kernel_Interpolation_for_Products_CUDA.ipynb

For large regression datasets (over 1,000,000 data points), GPyTorch supports exact GP inference by combining two key techniques:

1. BlackBox Matrix-Matrix Inference (BBMM): Computes the GP marginal log likelihood using only matrix multiplication. This stochastic approach allows scaling exact GPs to millions of points.
2. LanczOs Variance Estimates (LOVE): A technique used to rapidly speed up predictive variances and posterior sampling.

To implement these, refer to the Simple_GP_Regression_With_LOVE_Fast_Variances_and_Sampling.ipynb notebook.

GPyTorch provides specialized kernel implementations in gpytorch.kernels.keops that are compatible with the KeOps integration. These kernels allow for scalable exact Gaussian Processes by leveraging KeOps' symbolic autoregression and efficient kernel computations.

Supported kernels include:

• RBFKernel
• MaternKernel
• PeriodicKernel

Note that only specific standard kernels have KeOps implementations. If a required kernel is missing, you may consider contributing it via a pull request.

To implement Gaussian Process approximations in GPyTorch, you must use three composable objects in conjunction with a gpytorch.models.ApproximateGP model:

1. VariationalDistribution: Defines the form of the approximate inducing value posterior $q(\mathbf{u})$.
2. VariationalStrategy: Defines how to compute the variational distribution over function values $q(\mathbf{f}(\mathbf{X}))$ from the inducing point distribution $q(\mathbf{u})$.
3. gpytorch.mlls._ApproximateMarginalLogLikelihood: Defines the objective function (e.g., variational ELBO) used to learn the approximate posterior.

GPyTorch provides several standard mean functions for Gaussian Process modeling. These are located in the gpytorch.means module.

ZeroMean

ZeroMean is a mean function that assumes the mean of the process is zero.

ConstantMean

ConstantMean represents a mean function that is constant across the input space.

LinearMean

LinearMean represents a mean function that is a linear function of the input.

GPyTorch is available as an experimental package in the ArchLinux User Repository (AUR). Use an AUR helper like yay to install it.

yay -S python-gpytorch