DOCSEARCHbyAlgolia
Submit Docs

PatchTST Documentation

repository·main·Indexed 25 days ago

https://github.com/yuqinie98/patchtst

Official implementation of the 'A Time Series is Worth 64 Words' paper (ICLR 2023) for long-term time series forecasting using Transformers. The library features patching and channel-independence to improve efficiency and accuracy, supporting supervised learning, self-supervised pre-training, and fine-tuning. It includes a flexible Callback system for training lifecycles, distributed training utilities via DistributedTrainer, and implementations of FEDformer and Pyraformer.

Tokens
5.5K
Snippets
8
Records
45
Agent score
83%

What's inside PatchTST

  4. Run Supervised Learning training

    main

    Supervised learning scripts are located in ./scripts/PatchTST. The default model is PatchTST/42.

    To run a multivariate forecasting task for the weather dataset, execute the corresponding shell script. Results will be written to ./result.txt upon completion.

    Note: You can adjust hyperparameters such as patch length, look-back windows, and prediction lengths within the scripts.

    sh ./scripts/PatchTST/weather.sh

  5. Prepare data for Pyraformer

    main

    Datasets (Electricity, Wind, ETT, and App Flow) should be placed in the data directory.

    For single-step forecasting, you must preprocess the Electricity, Wind, and App Flow datasets using the provided scripts: preprocess_elect.py, preprocess_wind.py, and preprocess_flow.py respectively. Alternatively, you can download preprocessed data and place it in the data directory.

    To generate synthetic.npy, run the simulation script.

    python simulate_sin.py

  6. Set up FEDformer

    main

    To use FEDformer, ensure you have the following environment requirements met:

    1. Python: Version 3.6 or higher.
    2. PyTorch: Version 1.9.0.
    3. Data: Download the benchmark datasets required for your specific task.

    Once the environment is prepared and data is available, you can train the model using the provided experiment scripts.

    bash ./scripts/run_M.sh
bash ./scripts/run_S.sh

  7. Train FEDformer models using experiment scripts

    main

    FEDformer provides pre-configured experiment scripts for different benchmark types located in the ./scripts directory. Use these scripts to reproduce experimental results for multivariate or univariate time series.

    • For multivariate benchmarks, run: ./scripts/run_M.sh
    • For univariate benchmarks, run: ./scripts/run_S.sh
    bash ./scripts/run_M.sh
bash ./scripts/run_S.sh

  9. Use PatchTST for different task heads

    main

    The PatchTST model switches its output logic based on the head_type provided during initialization:

    Prediction

    • Goal: Time series forecasting.
    • Config: head_type="prediction".
    • Option: Set individual=True to use separate linear layers for each input variable, or False to use a shared layer.
    • Output: [bs x forecast_len x nvars].

    Regression

    • Goal: Continuous value prediction.
    • Config: head_type="regression".
    • Option: Provide y_range (a tuple) to apply SigmoidRange scaling to the output.
    • Output: [bs x target_dim].

    Classification

    • Goal: Categorical prediction.
    • Config: head_type="classification".
    • Output: [bs x target_dim] (where target_dim is the number of classes).

    Pretraining (Self-supervised)

    • Goal: Reconstruction task.
    • Config: head_type="pretrain".
    • Output: [bs x num_patch x n_vars x patch_len] (reconstructed patches).

  10. Fine-tune PatchTST (Self-supervised Learning)

    main

    After pre-training, use patchtst_finetune.py to perform either linear probing or full network fine-tuning on downstream tasks. You must provide the path or name of the pre-trained model.

    Example command for fine-tuning on the ettm1 dataset:

    python patchtst_finetune.py --dset ettm1 --pretrained_model <model_name>

  11. Pre-train PatchTST (Self-supervised Learning)

    main

    To perform self-supervised pre-training (specifically for PatchTST/64), use the patchtst_pretrain.py script. The trained model is saved to the saved_model folder for subsequent downstream tasks.

    Example command to run pre-training on the ettm1 dataset with a mask ratio of 0.4 using a single GPU:

    python patchtst_pretrain.py --dset ettm1 --mask_ratio 0.4

  12. Evaluate Pyraformer models

    main

    To evaluate a model, add the -eval flag to your execution command. Pretrained models should be placed in the models directory following the specific hierarchy for LongRange or SingleStep tasks.

    Long-range evaluation example: python long_range_main.py -data ETTh1 -input_size 168 -predict_step 168 -n_head 6 -eval

    Single-step evaluation example: python single_step_main.py -data_path data/elect/ -dataset elect -eval