Crossformer Documentation

repository·master·Indexed 20 days ago

https://github.com/thinklab-sjtu/crossformer

A Transformer-based model for multivariate time series forecasting. It utilizes Dimension-Segment-Wise (DSW) embedding and a Two-Stage Attention (TSA) layer to capture cross-dimension dependency. The repository includes scripts for training on standard and custom datasets via main_crossformer.py and evaluating models using eval_crossformer.py.

Tokens
1K
Snippets
3
Records
5
Agent score
22%

What's inside Crossformer

  1. Train and test Crossformer on standard datasets

    master

    To run experiments on standard datasets (like ETTh1), place the datasets in the datasets/ folder and execute main_crossformer.py. The model will automatically train and test, saving checkpoints to checkpoints/ and evaluation metrics to results/.

    python main_crossformer.py --data ETTh1 --in_len 168 --out_len 24 --seg_len 6 --itr 1
  2. Use custom multivariate time series data with Crossformer

    master

    To use your own data (e.g., AirQualityUCI.csv):

    1. Format the data: The CSV should have the first column as a date (or blank) and subsequent columns as the multivariate time series dimensions. Place the file in the datasets/ folder.
    2. Train: Run main_crossformer.py specifying the --data name, --data_path, and --data_dim (number of dimensions).
    3. Evaluate: Run eval_crossformer.py with the corresponding --setting_name.
    # Training with custom data
    python main_crossformer.py --data AirQuality --data_path AirQualityUCI.csv --data_dim 13 --in_len 168 --out_len 24 --seg_len 6
    
    # Evaluating custom data
    python eval_crossformer.py --setting_name Crossformer_AirQuality_il168_ol24_sl6_win2_fa10_dm256_nh4_el3_itr0 --save_pred
  3. Evaluate a trained Crossformer model

    master

    Use eval_crossformer.py to evaluate a specific model checkpoint. You must provide the --checkpoint_root and the exact --setting_name used during training.

    python eval_crossformer.py --checkpoint_root ./checkpoints --setting_name Crossformer_ETTh1_il168_ol24_sl6_win2_fa10_dm256_nh4_el3_itr0
  4. Reference: main_crossformer CLI arguments

    master

    The main_crossformer.py script is the primary entry point for training. Below are the available tuning parameters:

    ParameterDescription
    dataThe dataset name
    root_pathRoot path of the data file (default: ./datasets/)
    data_pathData file name (default: ETTh1.csv)
    data_splitTrain/Val/Test split ratio (e.g. 0.7,0.1,0.2) or absolute numbers (default: 0.7,0.1,0.2)
    checkpointsLocation to store trained models (default: ./checkpoints/)
    in_lenInput/history sequence length $T$ (default: 96)
    out_lenOutput/future sequence length $\tau$ (default: 24)
    seg_lenSegment length in DSW embedding $L_{seg}$ (default: 6)
    win_sizeNumber of adjacent segments to merge in HED (default: 2)
    factorNumber of routers in Cross-Dimension Stage $c$ (default: 10)
    data_dimNumber of dimensions $D$ (default: 7 for ETTh/ETTm)
    d_modelDimension of hidden states $d_{model}$ (default: 256)
    d_ffDimension of MLP in MSA (default: 512)
    n_headsNumber of heads in MSA (default: 4)
    e_layersNumber of encoder layers $N$ (default: 3)
    dropoutDropout probability (default: 0.2)
    num_workersData loader workers (default: 0)
    batch_sizeTraining/testing batch size (default: 32)
    train_epochsNumber of training epochs (default: 20)
    patienceEarly stopping patience (default: 3)
    learning_rateInitial optimizer learning rate (default: 1e-4)
    lradjLearning rate adjustment method (default: type1)
    itrNumber of experiment repetitions (default: 1)
    save_predSave predicted results as numpy arrays in results (default: False)
    use_gpuUse GPU (default: True)
    gpuGPU ID (default: 0)
    use_multi_gpuUse multiple GPUs (default: False)
    devicesDevice IDs for multi-GPU (default: 0,1,2,3)