Install Crossformer requirements
masterTo use Crossformer, ensure you have the following Python environment installed:
- Python 3.7.10
- numpy==1.20.3
- pandas==1.3.2
- torch==1.8.1
- einops==0.4.1
repository·master·Indexed 20 days ago
https://github.com/thinklab-sjtu/crossformerA Transformer-based model for multivariate time series forecasting. It utilizes Dimension-Segment-Wise (DSW) embedding and a Two-Stage Attention (TSA) layer to capture cross-dimension dependency. The repository includes scripts for training on standard and custom datasets via main_crossformer.py and evaluating models using eval_crossformer.py.
To use Crossformer, ensure you have the following Python environment installed:
To run experiments on standard datasets (like ETTh1), place the datasets in the datasets/ folder and execute main_crossformer.py. The model will automatically train and test, saving checkpoints to checkpoints/ and evaluation metrics to results/.
python main_crossformer.py --data ETTh1 --in_len 168 --out_len 24 --seg_len 6 --itr 1To use your own data (e.g., AirQualityUCI.csv):
datasets/ folder.main_crossformer.py specifying the --data name, --data_path, and --data_dim (number of dimensions).eval_crossformer.py with the corresponding --setting_name.# Training with custom data
python main_crossformer.py --data AirQuality --data_path AirQualityUCI.csv --data_dim 13 --in_len 168 --out_len 24 --seg_len 6
# Evaluating custom data
python eval_crossformer.py --setting_name Crossformer_AirQuality_il168_ol24_sl6_win2_fa10_dm256_nh4_el3_itr0 --save_predUse eval_crossformer.py to evaluate a specific model checkpoint. You must provide the --checkpoint_root and the exact --setting_name used during training.
python eval_crossformer.py --checkpoint_root ./checkpoints --setting_name Crossformer_ETTh1_il168_ol24_sl6_win2_fa10_dm256_nh4_el3_itr0The main_crossformer.py script is the primary entry point for training. Below are the available tuning parameters:
| Parameter | Description |
|---|---|
data | The dataset name |
root_path | Root path of the data file (default: ./datasets/) |
data_path | Data file name (default: ETTh1.csv) |
data_split | Train/Val/Test split ratio (e.g. 0.7,0.1,0.2) or absolute numbers (default: 0.7,0.1,0.2) |
checkpoints | Location to store trained models (default: ./checkpoints/) |
in_len | Input/history sequence length $T$ (default: 96) |
out_len | Output/future sequence length $\tau$ (default: 24) |
seg_len | Segment length in DSW embedding $L_{seg}$ (default: 6) |
win_size | Number of adjacent segments to merge in HED (default: 2) |
factor | Number of routers in Cross-Dimension Stage $c$ (default: 10) |
data_dim | Number of dimensions $D$ (default: 7 for ETTh/ETTm) |
d_model | Dimension of hidden states $d_{model}$ (default: 256) |
d_ff | Dimension of MLP in MSA (default: 512) |
n_heads | Number of heads in MSA (default: 4) |
e_layers | Number of encoder layers $N$ (default: 3) |
dropout | Dropout probability (default: 0.2) |
num_workers | Data loader workers (default: 0) |
batch_size | Training/testing batch size (default: 32) |
train_epochs | Number of training epochs (default: 20) |
patience | Early stopping patience (default: 3) |
learning_rate | Initial optimizer learning rate (default: 1e-4) |
lradj | Learning rate adjustment method (default: type1) |
itr | Number of experiment repetitions (default: 1) |
save_pred | Save predicted results as numpy arrays in results (default: False) |
use_gpu | Use GPU (default: True) |
gpu | GPU ID (default: 0) |
use_multi_gpu | Use multiple GPUs (default: False) |
devices | Device IDs for multi-GPU (default: 0,1,2,3) |