GPy Documentation

Because pickle is inconsistent across Python versions and class structures, do not use it for long-term model storage. Instead, save the model's param_array using numpy.save.

To load the model:

1. Recreate the model instance with initialize=False.
2. Call update_model(False) to avoid expensive algebra during loading.
3. Call initialize_parameter() to connect the parameters.
4. Load the saved parameters into the model using slice assignment (m[:] = ...).
5. Call update_model(True) to finalize the model state.

# let X, Y be data loaded above
# Model creation:
m = GPy.models.GPRegression(X, Y)
m.optimize()

# 1: Saving a model:
np.save('model_save.npy', m.param_array)

# 2: loading a model
# Model creation, without initialization:
m_load = GPy.models.GPRegression(X, Y, initialize=False)
m_load.update_model(False) # do not call the underlying expensive algebra on load
m_load.initialize_parameter() # Initialize the parameters (connect the parameters up)
m_load[:] = np.load('model_save.npy') # Load the parameters
m_load.update_model(True) # Call the algebra only once
print(m_load)

To extend GPy with new plotting capabilities, you should implement your plotting functions within the GPy.plotting hierarchy. All plotting-related code must reside in GPy.plotting or its submodules.

Implementation Steps

1. Create a module: Write your plotting function into a module under GPy.plotting.gpy_plot.<module_name>.
2. Use the plotting library: Access functionality via the GPy.plotting.plotting_library. It is recommended to use the pattern from . import plotting_library as pl and access methods via pl()..
3. Define parameters: The first argument of the plotting function must be self (to allow attaching the function to a class). Ensure you document all parameters, including kwargs for the specific backend.
4. Prepare data: Use helper_for_plot_data from .plot_util to generate the necessary grids (Xgrid, xx, yy) and handle dimension selection.
5. Create a canvas: Use pl().new_canvas(...) to initialize the plot. Always pass through kwargs to the new_canvas method to support backend-specific options.
6. Apply defaults: Use update_not_existing_kwargs(kwargs, pl().defaults.<default_name>) to ensure your plot respects the standard GPy plotting defaults for that specific backend and plot type.
7. Execute plotting: Use pl().plot(), pl().contour(), or pl().surface() depending on the dimensionality and projection, then return the result of pl().add_to_canvas(canvas, plots).
8. Attach to a class: To make the function available on a GPy object (e.g., a Kernel), inject it into the class in GPy.plotting.__init__.

Example of injecting a function into the Kern class:

from ..kern import Kern
Kern.plot_covariance = gpy_plot.kernel_plots.plot_covariance

from . import plotting_library as pl

def plot_covariance(kernel, x=None, label=None,
             plot_limits=None, visible_dims=None, resolution=None,
             projection=None, levels=20, **kwargs):
    # ... implementation ...
    canvas, kwargs = pl().new_canvas(projection=projection, xlabel=xlabel, ylabel=ylabel, zlabel=zlabel, **kwargs)
    # ...
    return pl().add_to_canvas(canvas, plots)

To create a custom model in GPy, you must inherit from the GPy.core.model.Model class. All models are built upon the GPy.core.parameterized.Parameterized base class, which handles parameter tying, bounding, fixing, and regex-based manipulation.

To ensure the model can be optimized and provide parameter introspection, you must implement three obligatory methods:

1. __init__(self, ...): Initialize the model and register parameters using self.add_parameter(param). Parameters should be created using GPy.Param (or Param).
2. log_likelihood(self): Return the log-likelihood of the model. For optimization tasks where you want to minimize a function, you should return the negative of that function.
3. parameters_changed(self): Update the internal state of the model and set the gradient for each parameter handle with respect to the log-likelihood. This is where you assign the analytical gradients to the parameter's .gradient attribute.

Below is a complete implementation of a custom model wrapping the Rosenbrock function.

from GPy import Model, Param
import scipy

class Rosen(Model):
    def __init__(self, X, name='rosenbrock'):
        super(Rosen, self).__init__(name=name)
        self.X = Param("input", X)
        self.add_parameter(self.X)

    def log_likelihood(self):
        return -scipy.optimize.rosen(self.X)

    def parameters_changed(self):
        self.X.gradient = -scipy.optimize.rosen_der(self.X)

After creating a model, you can verify the correctness of your analytical gradients using m.checkgrad(verbose=True). This compares the analytical gradients against numerical approximations. Once verified, you can use m.optimize() to find the optimal parameter values.

>>> m = Rosen(np.array([-1,-1]))
>>> m.checkgrad(verbose=True)
             Name           |     Ratio     |  Difference  |  Analytical   |  Numerical   
--------------------------------------------------------------------------------------------------
   rosenbrock.input[[0]]    |   1.000000    |   0.000000   |  -804.000000  |  -804.000000  
   rosenbrock.input[[1]]    |   1.000000    |   0.000000   |  -400.000000  |  -400.000000  
>>> m.optimize()
>>> print m.input
    Index  |  rosenbrock.input  |  Constraint  |   Prior   |  Tied to
     [0]   |        0.99999994  |              |           |    N/A    
     [1]   |        0.99999987  |              |           |    N/A