DOCSEARCHbyAlgolia
Submit Docs

NLPModels.jl

repository·main·Indexed 41 days ago

https://github.com/juliasmoothoptimizers/nlpmodels.jl

A standardized API and set of guidelines for representing non-linear programming (NLP) problems in Julia. It provides a consistent interface for optimization solvers to evaluate objectives, constraints, and their derivatives (gradients, Jacobians, and Hessians) through the AbstractNLPModel and NLPModelMeta structures. Includes support for Nonlinear Least Squares (NLS) models and a Batch API for processing multiple optimization problems simultaneously.

Tokens
9.7K
Snippets
14
Records
31
Agent score
52%

What's inside NLPModels.jl

  1. Available NLPModels API implementations

    main

    The NLPModels.jl ecosystem provides a unified interface for nonlinear optimization problems. Various packages implement this API to allow users to interface with different modeling languages, problem structures, or mathematical frameworks. Depending on your problem type, you should use one of the following specialized packages:

    • JuMP-based models: Use NLPModelsJuMP.jl if your problem is modeled using JuMP.jl.
    • AMPL-based models: Use AmplNLReader.jl for problems modeled using AMPL.
    • Automatic Differentiation: Use ADNLPModels.jl for models with built-in automatic differentiation.
    • Quadratic/Least Squares: Use QuadraticModels.jl for Linear Constraint Quadratic Programming (LCQP) or LLSModels.jl for linear least squares.
    • Structured Problems: Use BlockNLPModels.jl for block-structured nonlinear problems or PDENLPModels.jl for PDE-constrained problems.
    • External Libraries: Use CUTEst.jl for problems from the CUTEst collection.
    • Model Transformations: Use NLPModelsModifiers.jl to modify existing models (e.g., adding slack variables, moving constraints to the objective, or applying LBFGS Hessian approximations).

  2. What is an AbstractNLSModel?

    main

    An AbstractNLSModel is a special subtype of AbstractNLPModel used for Nonlinear Least Squares problems.

    In these models, the objective function $f(x)$ is specifically defined as the squared norm of a residual function: $f(x) = \frac{1}{2}\Vert F(x)\Vert^2$.

    Key features of AbstractNLSModel:

    • The residual function $F$ is explicitly available.
    • Individual values of the residual $F$ and its derivatives are accessible.

  3. Implement advanced counters for custom models

    main

    If your model does not store Counters in the standard counters field, you must implement the following interface to support the standard show and monitoring tools:

    1. get_counters(nlp): Returns the Counters object.
    2. neval_xxx(nlp): Accesses a specific field xxx of the Counters.
    3. reset!(nlp): Resets all counters.
    4. increment!(nlp, s): Increments counter s (where s is a Symbol).

    You can use the @default_counters macro to simplify this for wrapper models (like a SlackModel that wraps another model).

    Example for a wrapper model:

    @default_counters SlackModel model
    # Example of manual implementation for a wrapper
for counter in fieldnames(Counters)
  @eval begin
    $counter(nlp :: MyModel) = SOMETHING
  end
end

function reset!(nlp :: MyModel)
  RESET COUNTERS
end

function increment!(nlp :: MyModel, s :: Symbol)
  INCREMENT COUNTER s
end

  4. Access residual and derivative functions in NLSModels

    main

    For Nonlinear Least Squares (NLS) models, where the objective is $f(x) = \frac{1}{2} | F(x) |^2$, additional functions are available to access the residual $F(x)$ and its derivatives:

    Mathematical TermNLSModels Functions
    Residual $F(x)$residual, residual!
    Jacobian of Residual $J_F(x) = \nabla F(x)^T$jac_residual, jac_op_residual, jprod_residual, jtprod_residual, etc.
    Hessian of Residual $\nabla^2 F_i(x)$hess_residual, hess_op_residual, hprod_residual, jth_hess_residual, etc.

  5. How NLPModels represents optimization problems

    main

    NLPModels.jl uses a standardized representation for non-linear programming (NLP) problems. An optimization problem is represented as an instance or subtype of AbstractNLPModel.

    Every model is composed of:

    1. An instance of NLPModelMeta, which contains metadata about the problem (number of variables, constraints, bounds, etc.).
    2. Data specific to the provenance of the problem (the actual function definitions and parameters).

    The general problem form handled is: $$\begin{aligned} \min \quad & f(x) \ & c_L \leq c(x) \leq c_U \ & \ell \leq x \leq u \end{aligned}$$

    Where $f(x)$ is the objective, $c(x)$ are the constraints, and $x$ are the variables with bounds $\ell$ and $u$.

  6. Core NLPModels internal types and metadata structures

    main

    The NLPModels API is built around several core abstract types and metadata containers that define how optimization problems are represented and accessed. When building custom solvers or extending the API, you will interact with these primary abstractions:

    Abstract Model Types

    • AbstractNLPModel: The base type for nonlinear programming models.
    • AbstractNLSModel: The base type for nonlinear least squares models.

    Metadata Containers

    Metadata objects are used to store and retrieve problem information (like dimensions, objective functions, and constraints) efficiently. Key types include:

    • AbstractNLPModelMeta: The abstract base for NLP model metadata.
    • NLPModelMeta: The concrete metadata type for NLP models.
    • NLSMeta: The concrete metadata type for nonlinear least squares models.
    • nls_meta: A function or accessor related to NLSMeta.

  7. Use the Batch API in NLPModels.jl

    main

    The Batch API in NLPModels.jl allows for processing multiple optimization problems simultaneously. It reuses the same symbol names as the standard (non-batch) API, making it intuitive to switch between single-problem and batch-problem workflows.

    Key differences include:

    • Input arguments for variables (bx) and other vectors/matrices are typically AbstractMatrix types representing the batch.
    • The primary new in-place function is obj!, which returns a vector containing the objectives for every problem in the batch.

  8. How optimization problems are represented in NLPModels.jl

    main

    Optimization problems are represented by instances of a subtype of AbstractNLPModel.

    An AbstractNLPModel is composed of:

    1. An instance of NLPModelMeta: This provides metadata about the problem, such as the number of variables, constraints, and bounds.
    2. Problem-specific data: Data related to the provenance or specific details of the problem instance.

    This structure allows solvers to interact with a standardized API to evaluate functions and their derivatives regardless of how the model is internally defined.

  9. Reference NLPModels function naming conventions

    main

    When working with NLPModels, the function names follow specific suffixes to indicate their behavior and format:

    • ! (e.g., obj!, grad!): Performs the operation inplace.
    • _coord: Returns the value in coordinate format.
    • _dense: Returns the value in dense format.
    • prod: Performs a matrix-vector product.
    • _op: Returns an operator (compatible with LinearOperators.jl).
    • _lin / _nln: Refers to linear or nonlinear constraints respectively.

  10. Track function evaluations using NLPModel counters

    main

    The NLPModel object tracks the number of times various mathematical functions (objective, gradient, Jacobian, etc.) are evaluated during optimization. You can access these counts using specific counter functions.

    Some counters are universal, while others are specific to nonlinear least squares models (which include residual related counters).

    using ADNLPModels, LinearAlgebra, NLPModels
nlp = ADNLPModel(x -> dot(x, x), zeros(2))
for i = 1:100
   obj(nlp, rand(2))
end
neval_obj(nlp)

  11. Understand the NLPModels optimization problem format

    main

    NLPModels represents nonlinear optimization problems in the following standard format:

    $$\begin{aligned} \min \quad & f(x) \ & c_L \leq c(x) \leq c_U \ & \ell \leq x \leq u. \end{aligned}$$

    Where:

    • $f(x)$ is the objective function.
    • $c(x)$ represents the constraints (bounded by $c_L$ and $c_U$).
    • $x$ is the decision variable vector, bounded by $\ell$ and $u$.

  12. Create a minimal NLPModel

    main

    To create a new model in NLPModels.jl, your struct must derive from AbstractNLPModel (or another class derived from it).

    At a minimum, your model must include:

    1. A meta :: NLPModelMeta field to store problem information.
    2. A counters :: Counters field (or implement the advanced counter interface).

    Essential NLPModelMeta fields include:

    • nvar: Number of variables (mandatory).
    • x0: Starting point (defaults to zeros).
    • lvar, uvar: Variable bounds (defaults to (-∞,∞)).
    • ncon: Number of constraints (defaults to 0).
    • lcon, ucon: Constraint bounds (defaults to (-∞,∞)).
    • nnzh: Length of Hessian triplet vectors (defaults to nvar * (nvar + 1) / 2).
    • nnzj: Length of Jacobian triplet vectors (defaults to nvar * ncon).

    To specify linear constraints, use the lin keyword argument during NLPModelMeta initialization. The indices are then accessible via nlp.meta.lin.

    mutable struct MyModel{T, S} <: AbstractNLPModel{T, S}
  meta :: NLPModelMeta{T, S}
  counters :: Counters
end

function MyModel()
  return MyModel(NLPModelMeta(5), Counters())
end