epftoolbox

The epftoolbox is an open-access library designed for electricity price forecasting research, focusing on reproducibility and standardized benchmarks.

Key capabilities include:

• Forecasting Models: Access to state-of-the-art models including a Deep Neural Network and a LEAR (autoregressive model with LASSO regularization) model.
• Evaluation Metrics: Standard scalar metrics (MAE, sMAPE, MASE) and statistical tests (Diebold-Mariano and Giacomini-White) for comparing forecasting performance.
• Day-ahead Market Datasets: Access to five datasets (EPEX-BE, EPEX-FR, EPEX-DE, NordPool, and PJM) containing 6 years of historical prices and exogenous time series.
• Available Forecasts: Pre-computed forecasts from state-of-the-art methods for benchmarking new models without re-estimation.

The epftoolbox library is organized into three primary modules:

1. Data Management Module: Provides tools to manage, process, and obtain data for electricity price forecasting. It includes access to day-ahead electricity market data for EPEX-BE, EPEX-FR, EPEX-DE, NordPool, and PJM markets.
2. Models Module: Provides access to state-of-the-art forecasting methods for day-ahead electricity prices. Currently, it includes the learref and dnnref models, which are designed to be used automatically without expert knowledge.
3. Evaluation Module: Offers an interface for evaluating forecasts, including scalar metrics (e.g., MAE, MASE) and statistical tests to evaluate differences in forecasting performance.

The models subpackage provides interfaces for two state-of-the-art electricity price forecasting models:

1. LEAR model: Supports estimation, daily recalibration, and prediction.
2. DNN model: Supports estimation, hyperparameter optimization, daily recalibration, and prediction.

These models are designed for day-ahead electricity price forecasting based on methodologies described in Lago et al. (2021).

epftoolbox is an open-access library designed for electricity price forecasting research, focusing on reproducibility and research standards. The library is organized into three primary functional areas:

1. Data Management: Tools for processing data and extracting datasets.
2. Forecasting Models: Implementations of state-of-the-art models, specifically the learref and dnnref models.
3. Evaluation: Tools for assessing model performance using accuracy metrics and comparing forecasts through statistical testing.

The library is built on top of standard scientific Python libraries including scikit-learn, tensorflow, keras, hyperopt, statsmodels, numpy, and pandas.

The data subpackage provides tools for two primary workflows in electricity price forecasting:

1. Data Extraction: Use the data_extract module to automatically access data from five different day-ahead electricity markets or to implement an interface for reading data from other markets.
2. Data Wrangling: Use the data_wrangling module to apply common scaling transformations required for electricity price prediction models.

For specific models like LEAR and DNN, the subpackage also provides interfaces for estimation, daily recalibration, and prediction tasks.

The forecasts/ directory provides pre-computed forecasts generated by the two state-of-the-art models included in epftoolbox:

1. DNN (Deep Neural Network)
2. LEAR (Learning-based Error-Adjusting Regression)

These forecasts cover the five different day-ahead electricity markets supported by the toolbox. These datasets were used in the benchmark study: 'Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark' (Applied Energy 2021).

The epftoolbox accuracy metrics module provides an interface for evaluating electricity price forecasts. The available metrics are categorized into standard error metrics and scaled error metrics:

Standard Metrics:

• MAE (Mean Absolute Error)
• RMSE (Root Mean Square Error)
• MAPE (Mean Absolute Percentage Error)
• sMAPE (Symmetric Mean Absolute Percentage Error)

Scaled Error Metrics:

• MASE (Mean Absolute Scaled Error): Scales error by the in-sample MAE of a naive forecast.
• rMAE (Relative MAE): Scales error by the out-of-sample MAE of a naive forecast to allow for better comparisons across different time series and calibration windows.

The DNN module provides deep neural network models specifically tailored for electricity price forecasting. It is designed to optimize input features and hyperparameters automatically for different markets.

The module follows a two-step workflow:

1. Optimization: Use the hyperparameter_optimizer function to find the optimal set of hyperparameters and input features.
2. Deployment/Prediction: Use the DNN class, which consumes the results from the optimizer, to perform model recalibration and generate new predictions.

While you can use the base DNNModel class standalone, the DNN class is the recommended interface for production-like tasks as it handles the extraction of optimal parameters and manages recalibration before predictions.

The relative MAE (rMAE) is a metric used to normalize the Mean Absolute Error (MAE) by the MAE of a naive forecast. Unlike MASE (Mean Absolute Scaled Error), which relies on in-sample data and can lead to inconsistent comparisons when models use different calibration windows or rolling windows, rMAE uses the out-of-sample dataset to build the naive forecast. This makes it more suitable for non-stationary electricity price series and ensembles with varying calibration windows.

One-step ahead rMAE

For standard one-step ahead forecasting, rMAE is calculated as:

$$\mathrm{rMAE} = \frac{1}{N}\sum_{k=1}^{N}\frac{|p_k-\hat{p}k|}{\frac{1}{N-1}\sum{i=2}^{N} |p_i - p_{i-1} |}$$

The metrics module provides tools to evaluate forecast accuracy by calculating error values based on single metrics. These metrics analyze the distance between predictions and the actual values (such as the mean or median of real prices). Common examples include:

• MAPE (Mean Absolute Percentage Error)
• rMAE (Relative Mean Absolute Error)

Use these metrics when you need a single numerical value to represent the error magnitude of your predictions.

The LEAR model is a parameter-rich ARX model that uses LASSO for implicit feature selection. It is implemented as the LEAR class in epftoolbox.models.

To use the model, instantiate the LEAR class by providing the calibration window. The class provides four primary capabilities:

1. Recalibrate: Update the model parameters.
2. Predict: Generate forecasts based on the current state.
3. Recalibrate and Predict: Perform both steps in a single call.
4. Daily Recalibration and Prediction: A specialized function designed to work with pandas DataFrames for daily operational workflows.

MASE is a scaled error metric used to evaluate forecasting performance. It scales the Mean Absolute Error (MAE) by the in-sample MAE of a naive forecast.

Interpretation:

• A MASE value lower than 1 indicates the forecast is better than the average in-sample naive forecast.
• A MASE value higher than 1 indicates the forecast is worse than the average in-sample naive forecast.

Seasonal MASE: For seasonal time series, MASE can be calculated using a seasonal naive model in the denominator. For day-ahead price forecasting, common seasonal lengths ($m$) are:

• 24 (daily seasonality)
• 168 (weekly seasonality)