PyOptInterface

PyOptInterface is benchmarked based on its ability to generate optimization models and pass them to solvers. Performance is measured in terms of 'model construction time' (the time taken by the modeling interface to generate the model and hand it off to the optimizer).

Key performance characteristics observed in benchmarks:

• Comparison with direct solver APIs: PyOptInterface shows performance competitive with C++ interfaces and significantly faster than direct Python APIs like gurobipy or coptpy for large-scale models.
• Comparison with other modeling languages: It outperforms Pyomo by orders of magnitude in model construction time. It also shows significant performance advantages over linopy and cvxpy when generating and solving linear programming models.
• Scalability: The interface maintains high efficiency as the number of variables increases, from tens of thousands to millions of variables.

A callback is a user-defined Python function used to monitor or intervene in the optimization process (e.g., stopping when a condition is met or recording intermediate results). This is particularly useful for Mixed-Integer Programming (MIP) to control the branch-and-bound process.

Important Notes:

• Callback behavior depends heavily on the specific optimizer being used.
• Callback support is currently limited to Gurobi, COPT, and Xpress.
• PyOptInterface does not use a strictly unified API for callbacks because they are tightly coupled with the underlying solver; instead, it unifies common parts while aiming to expose all features available in the vendor's Python bindings.

Callback Function Signature: A callback function must accept two arguments:

1. model: The optimization model instance.
2. where: A flag indicating the stage of the optimization process (e.g., CallbackCodes for Gurobi, callback contexts for COPT, or specific callback points for Xpress).

def my_callback(model, where):
    # Implementation logic
    pass

PyOptInterface follows the 'Bring Your Own Container' (BYOC) principle. Instead of being restricted to specific container types, you can use standard Numpy ndarray objects to store variables and constraints. This allows you to leverage Numpy features like fancy-indexing and slicing to construct complex optimization models efficiently.

To use Numpy with PyOptInterface:

1. Initialize an ndarray with dtype=object to hold the variable objects.
2. Populate the array using model.add_variable().
3. Use Numpy slicing (e.g., x[i, :]) or methods (e.g., x.diagonal()) within poi.quicksum() to define constraints.

import numpy as np
import pyoptinterface as poi
from pyoptinterface import highs

model = highs.Model()
N = 8

# Create a 2D array of binary variables
x = np.empty((N, N), dtype=object)
for i in range(N):
    for j in range(N):
        x[i, j] = model.add_variable(domain=poi.VariableDomain.Binary)

# Use Numpy slicing to add constraints
for i in range(N):
    model.add_linear_constraint(poi.quicksum(x[i, :]), poi.Eq, 1.0)
    model.add_linear_constraint(poi.quicksum(x[:, i]), poi.Eq, 1.0)

model.optimize()

In PyOptInterface, nonlinear expressions cannot be constructed in the global scope. You must use the nl.graph() context manager to declare nonlinear expressions. This context allows PyOptInterface to trace the computational graph, enabling automatic differentiation for calculating gradients and Hessians.

Key Rules:

• All nonlinear expressions (using nl functions or operators) must be defined inside a with nl.graph(): block.
• Both model.add_nl_constraint() and model.add_nl_objective() must be called within the same nl.graph() context where the expressions were defined.
• For performance, especially in large-scale problems with repetitive structures, place the nl.graph() context inside loops so that PyOptInterface can recognize and accelerate the shared structure across iterations.

import pyoptinterface as poi
from pyoptinterface import nl, ipopt

model = ipopt.Model()
x = model.add_variable(name="x")

with nl.graph():
    # Nonlinear expressions must be inside this context
    z = nl.exp(x) * nl.pow(x, 2)
    model.add_nl_constraint(z <= 10.0)

To build large, maintainable optimization models in PyOptInterface, you should separate the model management from the problem data. The recommended pattern involves two components:

1. A model object: A thin wrapper around the optimizer API (e.g., highs.Model()) that manages variables, constraints, and the objective function. It does not store the problem state itself.
2. A container: A dict-like object to store references to variables and constraints. Using types.SimpleNamespace is recommended as it allows for easy attribute access (e.g., container.x).

You can decompose the modeling process by defining small, specialized functions that take a custom class (containing both the model and the container) as an argument. These functions can then be composed to build the full problem.

import types
import numpy as np
import pyoptinterface as poi
from pyoptinterface import highs

# 1. Define a class to hold the model and the container
class OptModel:
    def __init__(self):
        self.container = types.SimpleNamespace()
        self.model = highs.Model()

# 2. Define modular functions for variables and constraints
def declare_variables(m: OptModel, N):
    model = m.model
    x = np.empty((N, N), dtype=object)
    for i in range(N):
        for j in range(N):
            x[i, j] = model.add_variable(domain=poi.VariableDomain.Binary)
    m.container.x = x
    m.container.N = N

def add_constraints(m: OptModel):
    container = m.container
    x = container.x
    model = m.model
    N = container.N
    # Example: Row constraints
    for i in range(N):
        model.add_linear_constraint(poi.quicksum(x[i, :]), poi.Eq, 1.0)

# 3. Compose and solve
m = OptModel()
declare_variables(m, 8)
add_constraints(m)
m.model.optimize()

print("Termination status:", m.model.get_model_attribute(poi.ModelAttribute.TerminationStatus))