repository·main·Indexed 42 days ago
https://github.com/gurobi/gurobi-machinelearningA Python package (version 1.8dev1) that enables the integration of trained machine learning regression models from scikit-learn, Keras, PyTorch, and XGBoost into Gurobi optimization models via gurobipy. It allows developers to formulate ML predictions as constraints within a Mixed-Integer Programming (MIP) model using the core API function gurobi_ml.add_predictor_constr.
Gurobi Machine Learning is an open-source Python package designed to formulate trained regression models into a gurobipy model. This allows you to solve optimization problems that incorporate machine learning predictions directly within the Gurobi solver.
Supported ML Frameworks:
The package provides specialized APIs for integrating various machine learning frameworks into Gurobi models. Depending on your model type, you will use functions from one of the following modules:
gurobi_ml.sklearn)Used for standard Scikit-learn models and preprocessing, including:
linear_regressionlogistic_regressiondecision_tree_regressorrandom_forest_regressorgradient_boosting_regressormlpregressorpls_regressionpipelinecolumn_transformerpreprocessinggurobi_ml.keras)Used for Keras-based neural networks via the keras module.
gurobi_ml.torch)Used for PyTorch models, specifically supporting sequential models.
gurobi_ml.onnx)Used for models exported to the ONNX format via onnx_model.
gurobi_ml.xgboost)Used for XGBoost models via xgboost_regressor.
gurobi_ml.lightgbm)Used for LightGBM models via lgbm_regressor.
Gurobi Machine Learning is distributed under the Apache License 2.0.
Important Note on Gurobi Licensing:
While the gurobi-machinelearning package is Apache 2.0, the underlying Gurobi optimizer is commercial software and requires its own license.
gurobipy includes a limited-size license. This license is sufficient to solve all examples provided in the documentation.Logistic regression models are formulated using Gurobi's general function constraint. An intermediate free variable $\omega$ is created to represent the linear combination of inputs, and the output $y$ is expressed as $y = f(\omega)$, where $f$ is the logistic function.
Version-specific behavior:
FuncNonLinear to 1 for the logistic functions created by Gurobi Machine Learning.pwl_attributes keyword argument when adding the constraint.The package models dense neural networks using ReLU (Rectified Linear Unit) activations.
For each neuron, the relationship between inputs and outputs is formulated using Gurobi's max general constraint:
The neurons are then connected according to the network's topology.
Gurobi Machine Learning allows you to integrate trained regression models directly into a gurobipy.Model. In this setup, the input and the prediction of the ML model are treated as decision variables of the optimization problem. The model is formulated such that in any feasible solution, the output variables represent the values predicted by the ML model from the input variables ($y = g(x)$).
add_predictor_constrThe primary entry point is gurobi_ml.add_predictor_constr. It takes the following arguments:
gurobipy.Model instance.add_predictor_constr returns a modeling object (derived from AbstractPredictorConstr) that manages the variables and constraints added to the model. This object provides the following methods:
remove(): Removes all variables and constraints created to establish the relationship between input and output.print_stats(): Prints a summary of what was added to the model.get_error(): After Gurobi finds a solution, this computes the difference between the regression model's prediction (based on input values) and the actual values of the output variables in the Gurobi solution.# Conceptual usage pattern
import gurobi_ml
# model, predictor, x_vars, and y_vars must be defined
modeling_obj = gurobi_ml.add_predictor_constr(model, predictor, x_vars, y_vars)
# After solving...
error = modeling_obj.get_error()
modeling_obj.remove()Depending on your needs, use the following channels for support:
gurobipy Python API, contact Gurobi Support.Install the package using pip. This will also install gurobipy, numpy, and scipy if they are not already present in your environment.
Supported Environments:
Note that you only need to install specific machine learning libraries (like xgboost, lightgbm, or keras) if the regression model you intend to use requires them.
pip install gurobi-machinelearningDecision trees (and ensembles based on them, like Random Forest and Gradient Boosting) are formulated using binary decision variables for each leaf and indicator constraints for the split boundaries.
To control the accuracy and feasibility of these models, use the following keyword arguments in gurobi_ml.sklearn.add_decision_tree_regressor_constr:
epsilon (default 0): Approximates the strict inequality ($x_{i_v} > \theta_v$) used in right branches.epsilon must be greater than Gurobi's FeasibilityTol (default $10^{-6}$).epsilon too high creates a gap $[\theta_v, \theta_v + \epsilon]$ where no solution is feasible, potentially making the model infeasible if inputs are tightly constrained.safety_floor (default 0): Addresses issues where the split threshold $\theta_v$ is smaller than FeasibilityTol.safety_floor clamps thresholds to $\pm\text{safety_floor}$.safety_floor must be $\ge$ FeasibilityTol.The recommended way to install the package is using pip within a virtual environment. This command will automatically install the core dependencies: numpy, scipy, and gurobipy.
Note on Licensing: gurobipy is commercial software. While the pip installation includes a free limited license for testing, it can only solve models of limited size. For full-scale models, you must obtain a proper Gurobi license (e.g., via an academic program or commercial evaluation).
(.venv) pip install gurobi-machinelearningWhen using add_onnx_constr, the provided onnx.ModelProto is validated against several architectural requirements. If these are not met, a ModelConfigurationError is raised.
Validation Checks:
Add nodes are only valid if they follow a MatMul node and one of their inputs is a constant (initializer) representing the bias.Gemm, MatMul, Add (for bias), Relu, and Identity nodes are supported. Other operations will trigger a ModelConfigurationError.Gemm nodes must have alpha=1.0 and beta=1.0.The gurobi-machinelearning package provides a registry of supported machine learning frameworks that can be converted into Gurobi mathematical formulations. The library automatically detects which frameworks are available based on the modules currently loaded in your Python environment (sys.modules).
Supported frameworks include:
Pipeline objects.torch.nn.Sequential models.xgb.core.Booster and xgb.XGBRegressor.lgb.sklearn.LGBMRegressor.keras.Sequential and keras.Model.onnx.ModelProto.To see all currently available converters in your environment, you can call registered_predictors().
from gurobi_ml.registered_predictors import registered_predictors
# Get a mapping of model classes to their respective Gurobi conversion functions
all_convertors = registered_predictors()
print(all_convertors.keys())