DOCSEARCHbyAlgolia
Submit Docs

DCdetector: Dual Attention Contrastive Representation Learning

repository·main·Indexed 42 days ago

https://github.com/damo-di-ml/kdd2023-dcdetector

A dual attention contrastive representation learning model for time series anomaly detection. DCdetector utilizes a multi-scale dual-branch attention structure to learn permutation-invariant representations, enabling the discrimination between normal points and anomalies without relying on reconstruction loss. It supports Python 3.6 and PyTorch >= 1.4.0.

Tokens
1.4K
Snippets
3
Records
6
Agent score
5%

What's inside DCdetector

  1. Overview of DCdetector repository structure

    main

    The repository is organized into the following functional modules:

    • data_factory/: Preprocessing logic for all datasets.
    • dataset/: Contains the downloaded benchmark datasets.
    • main.py: The primary entry point where all model parameters can be adjusted.
    • metrics/: Implementation of evaluation metrics (e.g., VUC, affiliation precision/recall pair).
    • model/: Core DCdetector model architecture.
    • result/: Automatically saves training logs and experimental results.
    • scripts/: Shell scripts for running benchmark experiments and ablation studies.
    • solver.py: Handles the training, validation, and testing processes.
    • utils/: Utility functions for data processing and model construction.
    • requirements.txt: List of required Python packages.

  2. Train and evaluate DCdetector on benchmark datasets

    main

    DCdetector provides shell scripts in the ./scripts directory to reproduce experimental results for various benchmarks. You can run these scripts to automate the training and evaluation process for specific datasets.

    # Reproduce results for specific benchmarks
bash ./scripts/SMD.sh
bash ./scripts/MSL.sh
bash ./scripts/SMAP.sh
bash ./scripts/PSM.sh
bash ./scripts/SWAT.sh
bash ./scripts/NIPS_TS_Swan.sh
bash ./scripts/NIPS_TS_Water.sh
bash ./scripts/UCR.sh

# Run ablation experiments
bash ./scripts/Ablation_attention_head.sh
bash ./scripts/Ablation_encoder_layer.sh
bash ./scripts/Ablation_Multiscale.sh
bash ./scripts/Ablation_Window_Size.sh

  4. Execute training or testing via the main() function

    main

    The main(config) function initializes the Solver object with the provided configuration and triggers the requested pipeline mode.

    • If config.mode == 'train', it calls solver.train().
    • If config.mode == 'test', it calls solver.test().

    Note: The function requires a config object (typically an argparse.Namespace) that contains a model_save_path attribute and a mode attribute.

    from main import main
import argparse

# Mock config object
parser = argparse.ArgumentParser()
parser.add_argument('--mode', type=str, default='train')
parser.add_argument('--model_save_path', type=str, default='checkpoints')
config = parser.parse_args()

main(config)

  5. Configure DCdetector hyperparameters and execution

    main

    The following command-line arguments are used to configure the model architecture, training process, and hardware utilization.

    Model Architecture & Training

    • --win_size (int): Window size for time-series processing. Default: 100.
    • --patch_size (list): Size of patches. Default: [5].
    • --lr (float): Learning rate. Default: 1e-4.
    • --loss_fuc (str): Loss function to use (e.g., 'MSE'). Default: 'MSE'.
    • --n_heads (int): Number of attention heads. Default: 1.
    • --e_layers (int): Number of encoder layers. Default: 3.
    • --d_model (int): Model dimension. Default: 256.
    • --rec_timeseries (flag): Whether to reconstruct the time series. Default: True.
    • --num_epochs (int): Number of training epochs. Default: 10.
    • --batch_size (int): Batch size for training. Default: 128.
    • --anormly_ratio (float): Anomaly ratio. Default: 4.00.

    Data Configuration

    • --dataset (str): Dataset type (e.g., 'credit', 'UCR', 'UCR_AUG', 'SMD_Ori'). Default: 'credit'.
    • --index (int): Dataset index. Default: 137.
    • --data_path (str): Path to the dataset file. Default: './dataset/creditcard_ts.csv'.
    • --input_c (int): Input channels. Default: 9.
    • --output_c (int): Output channels. Default: 9.
    • --mode (str): Execution mode, either 'train' or 'test'. Default: 'train'.
    • --model_save_path (str): Directory to save model checkpoints. Default: 'checkpoints'.

    Hardware & GPU Configuration

    • --use_gpu (bool): Enable GPU usage. Default: True.
    • --gpu (int): Primary GPU ID. Default: 0.
    • --use_multi_gpu (flag): Enable multi-GPU training. Default: True.
    • --devices (str): Comma-separated list of device IDs for multi-GPU. Default: '0,1,2,3'.

  6. Run DCdetector via CLI

    main

    The main.py script serves as the entrypoint for the DCdetector pipeline. You can execute training or testing by passing various configuration flags via the command line. The script supports two modes: train (default) and test.

    To run the pipeline, use python main.py followed by your desired arguments. The script will automatically handle directory creation for model_save_path and log outputs to a file in the result/ directory named after your data_path.

    python main.py --mode train --dataset credit --data_path ./dataset/creditcard_ts.csv --model_save_path ./my_checkpoints