DOCSEARCHbyAlgolia
Submit Docs

PyTorch Forecasting

repository·main·Indexed 26 days ago

https://github.com/sktime/pytorch-forecasting

A high-level library for deep learning time series forecasting using PyTorch Lightning. It provides specialized dataset abstractions, dataloaders, normalizers, and state-of-the-art neural architectures including Temporal Fusion Transformers (TFT), N-BEATS, N-HiTS, and DeepAR. The library includes extension templates for implementing custom v1 and v2 neural network models and data modules, as well as utilities like check_estimator for verifying model compatibility.

Tokens
17.9K
Snippets
27
Records
69
Agent score
89%

What's inside pytorch-forecasting

  2. Identify supported models in pytorch-forecasting

    main

    Models in pytorch-forecasting are categorized into two versions:

    1. v1 Models: These are part of the stable version and are suitable for production use.
    2. v2 Models: These are part of the experimental, unstable version. Use these models with caution as they may undergo changes or be unstable.

    Refer to the specific version documentation (v1 or v2) to see the full list of available model architectures.

  3. Understand the API-v2 two-layer data architecture

    main

    The API-v2 data pipeline uses a two-layer architecture to separate data ingestion from preprocessing and loading:

    1. D1 Layer (Dataset): Ingests raw tabular data (e.g., pandas DataFrames) and converts it into torch tensors. It extracts base-level metadata like static variables and time series properties but does not handle complex preprocessing or batching.
    2. D2 Layer (DataModule): A PyTorch Lightning LightningDataModule that handles preprocessing (normalizers/encoders), batching (creating train_dataloader, val_dataloader, and test_dataloader), and collecting metadata required for model initialization (e.g., embedding sizes, vocabulary states).

    Warning: The v2 modules are currently in active development and are in beta. Use this API with caution.

  4. Understand the PyTorch Forecasting API v2 Architecture

    main

    API v2 is currently in beta and active development. It is designed to decouple models from data structures, allowing models to interface more easily with standard PyTorch tensors and data loaders.

    The architecture follows a four-layered structure:

    1. D1 Layer (Dataset Layer): Ingests raw data, converts it to PyTorch tensors, and extracts metadata (like static variables).
    2. D2 Layer (DataModule Layer): A LightningDataModule that handles preprocessing, instantiates dataloaders, and collects structural information (e.g., number of categorical variables).
    3. Model Layer: A LightningModule containing the pure PyTorch implementation of forecasting algorithms, agnostic of data ingestion complexities.
    4. Package Layer: A high-level wrapper that orchestrates the workflow, manages layer initialization, and exposes unified fit() and predict() interfaces.

  5. Understand the V2 Architecture (M Layer and P Layer)

    main

    The V2 ecosystem uses a modular architecture to decouple algorithmic logic from data processing:

    • The M Layer (Model): Core torch neural network implementations that inherit from PyTorch Lightning's LightningModule. This layer is intended for advanced users who want to build custom training, testing, and prediction pipelines using pure PyTorch.
    • The P Layer (Package): A high-level, sklearn-like orchestration layer. It wraps the M Layer to provide a simplified interface for fast training, prediction, and checkpointing. It uses configuration dictionaries to manage data, models, and trainers, allowing you to use .fit() and .predict() without writing boilerplate PyTorch code.

  6. Available Forecasting Models

    main

    PyTorch Forecasting provides several state-of-the-art deep learning architectures for time series forecasting:

    • Temporal Fusion Transformers (TFT): Interpretable multi-horizon forecasting.
    • N-BEATS: Neural basis expansion analysis for interpretable forecasting.
    • N-HiTS: Neural Hierarchical Interpolation, supports covariates and is suited for long-horizon forecasting.
    • DeepAR: Probabilistic forecasting with autoregressive recurrent networks.
    • Standard Networks: LSTM, GRU, and MLP on the decoder for baselining.
    • Baseline Model: A simple model that always predicts the latest known value.

  7. Implement a new Model and Package

    main

    To implement a new model, you must provide both a Model class and a Package class to separate ML logic from framework metadata.

    1. Package Configuration (model_pkg.py)

    Inherit from Base_pkg. You must implement:

    • get_cls(): Imports and returns your MyModel class.
    • get_datamodule_cls(): Imports and returns a compatible PyTorch Forecasting datamodule class. Every model must link to at least one compatible data module.
    • get_test_train_params(): Returns a list of dictionaries for CI testing. The first element must be an empty dict {} to test default parameters.
    • _tags: A dictionary defining integration rules. Key tags include:
      • info:name: Human-readable name.
      • info:pred_type: e.g., ["point"], ["quantile"], ["distr"].
      • info:y_type: e.g., ["numeric"], ["category"].
      • info:compute: Integer (1 to 5) representing compute intensity.
      • capability:exogenous: Boolean for exogenous variable support.
      • capability:multivariate: Boolean for multivariate target support.
      • capability:pred_int: Boolean for prediction interval support.
      • capability:flexible_history_length: Boolean for variable-length history support.
      • capability:cold_start: Boolean for predictions with little/no history.

    2. Model Configuration (model.py)

    Inherit from BaseModel. You must implement:

    • __init__(): Initialize network components. Must call self.save_hyperparameters() and super().__init__().
    • _pkg(): A class method that imports and returns your MyModel_pkg class.
    • forward(x: dict[str, torch.Tensor]) -> dict[str, torch.Tensor]: The PyTorch forward pass logic.

  8. Implement a new Data Module and Dataset

    main

    Implement a custom data module only if existing modules in pytorch_forecasting.data.data_module cannot format your data into the required inputs (e.g., you need custom metadata preparation or specialized preprocessing).

    1. Data Module (data_module.py)

    Inherit from LightningDataModule. You must implement:

    • _prepare_metadata(): Derives metadata required for model initialization from raw data/parameters.
    • metadata: A property that returns _metadata, invoking _prepare_metadata() if it is null.
    • _preprocess_data(series_idx): Logic to transform raw series before dataset consumption.
    • setup(stage: str): Standard Lightning method to instantiate train/val/test splits.

    2. Dataset (_dataset.py)

    Inherit from torch.utils.data.Dataset. You must implement:

    • __init__(data_module, ...): Accepts the parent Data Module to read preprocessed data and states.
    • __getitem__(idx): Returns the processed item dictionary exactly as required by the model's forward pass.

  10. Install PyTorch Forecasting

    main

    You can install pytorch-forecasting using pip or conda.

    Windows Users: You must first install PyTorch specifically for Windows:

    pip install torch -f https://download.pytorch.org/whl/torch_stable.html

    Standard Installation:

    pip install pytorch-forecasting

    Conda Installation: Install pytorch-forecasting from conda-forge and pytorch from the pytorch channel:

    conda install pytorch-forecasting pytorch -c pytorch>=1.7 -c conda-forge

    With MQF2 Loss Support: To use the multivariate quantile loss (MQF2), install the extra dependency:

    pip install pytorch-forecasting[mqf2]
    pip install pytorch-forecasting

  11. Use the M Layer (Model) for custom PyTorch Lightning pipelines

    main

    If you require full control over the training process, you can bypass the Package layer and use the M Layer directly. This involves manually setting up a TimeSeries dataset, an EncoderDecoderTimeSeriesDataModule, the model (e.g., TFT), and a PyTorch Lightning Trainer.

    Note: When initializing the model manually, you must pass metadata=data_module.metadata to ensure important parameters like encoder_cont are correctly initialized.

    from pytorch_forecasting.data.timeseries import TimeSeries
from pytorch_forecasting.data.data_module import EncoderDecoderTimeSeriesDataModule
from pytorch_forecasting.metrics import MAE, SMAPE
from pytorch_forecasting.models.temporal_fusion_transformer._tft_v2 import TFT
from lightning.pytorch import Trainer

# Create TimeSeries dataset
dataset = TimeSeries(
    data=data_df,
    time="time_idx",
    target="y",
    group=["series_id"],
    num=["x", "future_known_feature", "static_feature"],
    cat=["category", "static_feature_cat"],
    known=["future_known_feature"],
    unknown=["x", "category"],
    static=["static_feature", "static_feature_cat"],
)

# Create the data_module
data_module = EncoderDecoderTimeSeriesDataModule(
    time_series_dataset=dataset,
    max_encoder_length=30,
    max_prediction_length=1,
    batch_size=32,
)

# Initialise the Model
model = TFT(
    loss=MAE(),
    logging_metrics=[MAE(), SMAPE()],
    optimizer="adam",
    optimizer_params={"lr": 1e-3},
    lr_scheduler="reduce_lr_on_plateau",
    lr_scheduler_params={"mode": "min", "factor": 0.1, "patience": 10},
    hidden_size=64,
    num_layers=2,
    attention_head_size=4,
    dropout=0.1,
    metadata=data_module.metadata,  # pass the metadata from the datamodule to the model
)

# Train the model
trainer = Trainer(
    max_epochs=5,
    accelerator="auto",
    devices=1,
    enable_progress_bar=True,
    log_every_n_steps=10,
)

trainer.fit(model, data_module)

  12. Set up an editable development environment

    main

    For contributing to the project, it is recommended to use a conda environment and an editable install.

    1. Create and activate a new environment with a supported Python version (3.10 - 3.14).
    2. Install the package in editable mode with developer dependencies using pip install -e ".[dev]".
    3. To include all soft dependencies (extras) along with dev dependencies, use pip install -e ".[all_extras,dev]".