SysIdentPy Documentation

The sysidentpy.metrics module provides 13 public functions for evaluating regression and forecasting errors by comparing observed values (y) with predicted values (yhat).

Metrics are categorized by their mathematical properties:

• Signed error: forecast_error, mean_forecast_error. Useful for detecting bias/direction.
• Squared error: mean_squared_error, root_mean_squared_error. Penalizes large errors heavily.
• Normalized squared error: normalized_root_mean_squared_error, root_relative_squared_error. Dimensionless; used for scale comparison.
• Absolute error: mean_absolute_error, median_absolute_error. Direct interpretation; less sensitive to outliers.
• Scaled absolute error: mean_absolute_scaled_error. Compares MAE against a naive forecast from training data.
• Logarithmic/Percentage error: mean_squared_log_error, symmetric_mean_absolute_percentage_error. Assesses relative differences.
• Goodness of fit: explained_variance_score, r2_score. Compares error to output variability.

import numpy as np
from sysidentpy.metrics import (
    explained_variance_score,
    forecast_error,
    mean_absolute_error,
    mean_absolute_scaled_error,
    mean_forecast_error,
    mean_squared_error,
    mean_squared_log_error,
    median_absolute_error,
    normalized_root_mean_squared_error,
    r2_score,
    root_mean_squared_error,
    root_relative_squared_error,
    symmetric_mean_absolute_percentage_error,
)

y = np.array([3.0, -0.5, 2.0, 7.0])
yhat = np.array([2.5, 0.0, 2.0, 8.0])
y_train = np.array([1.0, 2.0, 3.0, 4.0])

# Example usage of various metrics
errors = forecast_error(y, yhat)
bias = mean_forecast_error(y, yhat)
mse = mean_squared_error(y, yhat)
rmse = root_mean_squared_error(y, yhat)
nrmse = normalized_root_mean_squared_error(y, yhat)
rrse = root_relative_squared_error(y, yhat)
mae = mean_absolute_error(y, yhat)
median_ae = median_absolute_error(y, yhat)
mase = mean_absolute_scaled_error(y, yhat, y_train)
msle = mean_squared_log_error(y, yhat)
smape = symmetric_mean_absolute_percentage_error(y, yhat)
evs = explained_variance_score(y, yhat)
r2 = r2_score(y, yhat)

SysIdentPy is a framework for building Dynamical Nonlinear Models for time series and dynamic systems using NARMAX (Nonlinear AutoRegressive Moving Average with eXogenous inputs) models and their variants (NARX, NAR, NARMA, NFIR, ARMAX, ARX, ARMA, etc.).

Key Capabilities:

• Model Structure Selection: Methods like FROLS, MetaMSS, AOLS, UOFR, Entropic Regression, RMSS, and Orthogonal Floating Search (OSF, OIF, OOS/O2S).
• Basis Functions: Support for up to 8 different basis functions (linear and nonlinear) that can be ensembled.
• Parameter Estimation: Over 15 methods for parameter estimation, including multiobjective estimation using affine information.
• Model Simulation: Use the SimulateNARMAX class to reproduce results from literature or test published models.
• Neural NARX: Integration with PyTorch to create custom neural NARX architectures using PyTorch optimizers and loss functions.
• General Estimators: Compatibility with interfaces from scikit-learn, Catboost, and other compatible tools.

NARMAX stands for Non-linear Autoregressive Models with Moving Average and Exogenous Input. It is a class of mathematical models used to represent complex nonlinear systems by relating the current output to past inputs, past outputs, and noise/uncertainty terms.

A NARMAX model is defined by the equation:

$$y_k= F^\ell[y_{k-1}, \dots, y_{k-n_y},x_{k-d}, x_{k-d-1}, \dots, x_{k-d-n_x}, e_{k-1}, \dots, e_{k-n_e}] + e_k$$

Key components:

• $y_k$: System output at discrete time $k$.
• $x_k$: System input.
• $e_k$: Uncertainties or noise.
• $n_y, n_x, n_e$: Maximum lags for output, input, and noise respectively.
• $\ell$: Nonlinearity degree.
• $d$: Time delay (typically $d=1$).

Common Variants:

• NARX: NARMAX without noise terms ($e_{k-n_e}$).
• NARMA: NARMAX where $\ell > 1$ and there are no input terms.
• NAR: NARMAX with no input or noise terms.
• ARMAX/ARX: Linear versions where $\ell = 1$.

System Identification (SI) is a data-driven framework used to model dynamical systems. SysIdentPy is a library designed to facilitate the modeling of nonlinear dynamic systems, specifically focusing on the NARMAX (Nonlinear AutoRegressive Moving Average model with eXogenous inputs) method.

SysIdentPy provides tools to interact with almost every step of the NARMAX modeling process, except for control design. The supported steps include:

1. Model Representation: Defining the mathematical form.
2. Model Structure Selection (MSS): Determining which terms belong in the final model (the most critical and complex step).
3. Parameter Estimation: Estimating coefficients for the selected terms.
4. Model Validation: Ensuring the model is unbiased and accurate.
5. Model Prediction/Simulation: Predicting future outputs or simulating system behavior.
6. Analysis: Understanding the dynamical properties of the system.