GPBoost Documentation

GPBoost is a software library designed to combine tree-boosting with Gaussian process and grouped random effects models (also known as mixed effects models or latent Gaussian models).

Beyond combined modeling, GPBoost supports the independent application of:

• Tree-boosting
• Gaussian process models
• (Generalized) linear mixed effects models (LMMs and GLMMs)

The {fmt} library is a high-performance C++ formatting library designed to provide a modern, safe, and fast alternative to printf and iostreams. It combines the speed and concise syntax of printf with the type safety and support for user-defined types found in iostreams.

Key advantages include:

• Performance: Faster than Boost Format and iostreams.
• Safety: Type-safe formatting that avoids common printf pitfalls.
• Features: Supports positional arguments (useful for i18n), leading zeros, octal/hexadecimal encoding, and runtime width/alignment specification—features missing in some other fast libraries like FastFormat.
• Syntax: Uses a Python-inspired format string syntax.

The double-conversion library provides efficient binary-decimal and decimal-binary routines for IEEE doubles. These routines were originally extracted from the V8 JavaScript engine and refactored for general use.

For detailed API documentation and usage patterns, refer to the following headers:

• double-conversion/string-to-double.h
• double-conversion/double-to-string.h

Code examples for conversions can be found in test/cctest/test-conversions.cc.

Boost.Compute is a C++ GPU/parallel-computing library built on top of OpenCL. It provides two main layers of abstraction:

1. Core Library: A thin C++ wrapper over the OpenCL API used to manage compute devices, contexts, command queues, and memory buffers.
2. Generic Interface: An STL-like interface that provides common algorithms (e.g., transform(), accumulate(), sort()), containers (e.g., vector<T>, flat_set<T>), parallel-computing extensions (e.g., exclusive_scan(), scatter(), reduce()), and specialized iterators (e.g., transform_iterator<>, permutation_iterator<>, zip_iterator<>).

The MATLAB/ directory contains the MATLAB interface, including mex-files for high performance and MATLAB wrappers for ease of use.

• Interface Location: MATLAB/CSparse contains the core wrappers (e.g., cs_add.m, cs_chol.m).
• Demos: MATLAB/Demo provides MATLAB-specific demonstrations.
• Tests: MATLAB/Test contains extensive testing suites for the library's functionality.

The repository provides several specialized examples for different modeling tasks:

• GPBoost and LaGaboost algorithms: Demonstrates combining tree-boosting with Gaussian process and random effects models for both Gaussian data (regression) and non-Gaussian data (classification).
• Parameter tuning: Shows how to perform hyperparameter optimization using deterministic or random grid search.
• Generalized linear Gaussian process and mixed effects models: Examples of modeling using generalized linear frameworks combined with GP and mixed effects.
• Panel data application: Demonstrates how to apply the GPBoost algorithm specifically to panel data structures.

GPBoost (via the LightGBM algorithm) supports categorical features directly. This is more efficient and accurate than one-hot encoding, especially for high-cardinality features, as it avoids creating deep, unbalanced trees.

Instead of one-hot encoding, GPBoost partitions categories into two subsets by sorting them according to the training objective. To use this, you must specify the feature indices in the categorical_feature parameter.

When using the cpu device type, you can manually control how histograms are built to optimize performance or memory usage. By default, GPBoost tries both methods and selects the faster one, but setting these manually removes the testing overhead.

Use force_col_wise=true when:

• The number of columns is large, or the total number of bins is large.
• num_threads is large (e.g., > 20).
• You want to reduce memory cost.

Use force_row_wise=true when:

• The number of data points is large and the total number of bins is relatively small.
• num_threads is relatively small (e.g., <= 16).
• You want to use small bagging_fraction or goss boosting to speed up training.
• Warning: This doubles the memory cost for the Dataset object.

Note: You cannot use both at the same time; choose only one.

The core GPBoost API is built around three primary entities:

• GPModel: The main model object used for training and prediction.
• Booster: Represents the tree-boosting component of the model.
• Dataset: The specialized data structure used to hold features, labels, and weights for training and evaluation.

import gpboost

# Typical workflow involves creating a Dataset, then training a GPModel
dataset = gpboost.Dataset(X, label=y)
model = gpboost.GPModel(params)
model.fit(dataset)

For large datasets, you can use scalable Gaussian process (GP) approximations via the gp_approx argument to reduce computational demand.

• Vecchia approximations: Generally recommended. Use gp_approx = "vecchia". The num_neighbors parameter controls the trade-off between runtime and accuracy (smaller values are faster).
• VIF (Vecchia-Inducing-Points Full-Scale) approximations: Recommended for higher-dimensional inputs (e.g., > 10). Use gp_approx = "vif". The num_neighbors and num_ind_points parameters control the trade-off between runtime and accuracy (smaller values are faster).