NeuroMANCER

SLiM (Structured Linear Maps) is a package providing a suite of structured linear maps designed as drop-in replacements for PyTorch's nn.Linear module.

These parametrizations are intended to:

• Enhance learning stability.
• Embed inductive priors that encode domain-specific knowledge.
• Facilitate exploration of neural networks from a dynamical systems perspective.

All structured maps in this package share a common API to ensure they can be easily swapped into existing PyTorch architectures.

Neuromancer supports differentiable system identification (System ID) methods for both discrete-time and continuous-time dynamical systems. This allows for learning unknown parameters $\theta$ of a model $f_{\theta}$ from observational data using gradient-based optimization.

Supported methods include:

• Neural Ordinary Differential Equations (NODEs): Continuous-time models.
• Neural State Space Models (NSSMs): Discrete-time models.
• Universal Differential Equations (UDEs): Hybrid models incorporating prior physical knowledge.

For continuous systems, Neuromancer uses ODE solvers (like Euler or Runge-Kutta) to obtain a discretized system $x_{k+1} = \text{ODESolve}(f_{\theta}(x_k))$ that can be trained.

Differentiable Parametric Programming (DPP) is a method used to learn solutions (explicit solvers) to constrained optimization problems where the solution $x$ depends on varying problem parameters $\xi$.

Neuromancer implements DPP by allowing you to formulate optimization problems within a differentiable computational graph. By leveraging PyTorch's automatic differentiation (AD), you can compute the sensitivities of constrained optimization problems with respect to their parameters. This enables the use of gradient-based optimizers (like Adam or SGD) to train a neural network (a 'solution map') to approximate the optimal solution.

Key Advantage: Compared to classical online solvers (e.g., IPOPT), a trained DPP model provides significantly faster online evaluation, often by orders of magnitude, making it suitable for real-time applications.

NeuroMANCER uses a symbolic programming paradigm to bridge machine learning and scientific computing. The core workflow involves:

1. Defining Modules: Using nm.modules (like MLP) to define neural architectures.
2. Symbolic Wrapping: Wrapping modules in a nm.system.Node to define input/output relationships (e.g., map(p) -> x).
3. Variables and Constraints: Using nm.constraint.variable to define decision variables and parameters, and defining constraints using standard mathematical syntax.
4. Objectives: Defining objective functions via .minimize() on symbolic expressions.
5. Loss and Problem Construction: Combining objectives and constraints into a nm.loss.PenaltyLoss and passing them along with Node objects into an nm.problem.Problem to create a fully differentiable optimization task.

NeuroMANCER utilizes a Node and System Pattern as its core abstraction:

• Nodes: Represent individual computational elements.
• Systems: Represent composed collections of nodes.

This is complemented by Symbolic Programming, which allows users to define constraints, objectives, and optimization problems using symbolic representations that interface directly with the Node class.

When learning a solution map $\pi_{\Theta}(\xi)$ for a differentiable constrained optimization problem, you must enforce constraints during training. Neuromancer supports several methods to incorporate constraints into the loss function:

1. Penalty Loss: The simplest approach, which augments the objective loss by penalizing constraint violations. Use neuromancer.loss.PenaltyLoss for this.
2. Barrier Loss: Uses barrier functions to prevent the solution from leaving the feasible region.
3. Augmented Lagrangian: A more sophisticated method for handling constraints.

All these methods allow the loss function to be differentiated, enabling end-to-end training of the solution map.

NeuroMANCER is designed to solve three primary types of problems using its Problem class wrapper:

• Learning to Optimize (L2O): Focuses on parametric programming. Users interface with Modules (specifically Node and Variable classes) and construct constraints by composing symbolic variables.
• Learning to Model (L2M): Focuses on modeling dynamics (e.g., ODEs, PDEs, KANs). Users use neural blocks (found in modules/blocks.py) to model data dynamics. Supported methods include neural state space models, neural ODEs, neural SDEs, PINNs, and KANs.
• Learning to Control (L2C): Focuses on control methodologies, specifically Differential Predictive Control. Users combine L2M modeling (system identification) with a neural control block in a closed-loop configuration, often incorporating constraints similar to L2O.

When using the NODE implementation in the train.py benchmark script, the following default settings are applied to the model:

• Architecture: 4-layer neural network with 128 nodes per layer modeling the right-hand side of the ODE.
• Observability: Fully observable state space.
• Integration: Euler integration method.
• Time Step: Inferred from the PSL (Physics-based Symbolic Language) system.

Users can ingest data or generate it using several entry points:

• PSL (Physics Simulation Library): Provides ready-to-use, physics-based models for fast setup of canonical problems (especially in controls), though it offers less flexibility than custom models.
• Dynamics/ode.py: Provides ODESystem, a flexible base for building custom ODE models and performing grey-box or physics-informed neural ODE modeling.
• DictDataset API: For custom data ingestion (e.g., L2M/L2C workflows), data should be converted to the DictDataset format.