DOCSEARCHbyAlgolia
Submit Docs

EasyTPP: Temporal Point Process Toolkit

repository·main·Indexed 43 days ago

https://github.com/ant-research/easytemporalpointprocess

A PyTorch-based toolkit for Temporal Point Process (TPP) research. It provides modular implementations of state-of-the-art models—including NHP, THP, SAHP, and S2P2—along with standardized datasets and a pipeline for hyper-parameter optimization using optuna. EasyTPP facilitates customized development through a BaseModel architecture and provides tools for log-likelihood computation, event sampling via thinning algorithms, and benchmarking.

Tokens
14.2K
Snippets
36
Records
64
Agent score
53%

What's inside EasyTPP

  1. Overview of EasyTPP

    main

    EasyTPP is a development and application toolkit for Temporal Point Processes (TPP). It is designed for researchers and practitioners to facilitate customized development and open benchmarking of TPP models.

    Key features include:

    • Configurability: Modularized models with abstract classes for custom TPP development.
    • PyTorch-based: Implements state-of-the-art models using PyTorch 1.7.0+.
    • Reproducibility: Provides easy reproduction of all benchmarks.
    • Hyper-parameter optimization: Includes a pipeline based on optuna for HPO.

  2. Overview of the EasyTPP Framework

    main

    EasyTPP is an open-source PyTorch library designed for Neural Temporal Point Processes (TPP). It provides a fully automated pipeline for model training and prediction. The framework is composed of loosely-coupled modules to allow for customization:

    • Preprocess module: Handles data batch-wise padding, inter-time processing, and raw sequence preparation.
    • Model module: Contains implementations of various State-of-the-Art (SOTA) TPP models.
    • Config module: Manages the construction of configuration objects required to run the pipeline.
    • Runner module: Orchestrates the execution of the training and prediction pipelines.

  3. Understand the Basic Model Structure in EasyTPP

    main

    EasyTPP uses an inheritance-based architecture for its models. All concrete model classes inherit from BaseModel, which is located in basemodel.py. This base class provides shared utilities for:

    • Likelihood computation: Shared methods for calculating log-likelihood.
    • Sampling: Utilities for generating event sequences.
    • Generation: Tools for model-driven sequence generation.

    When implementing a new model, you should inherit from BaseModel to leverage these core functionalities or override them if your model requires a custom objective function.

  4. How the Thinning Algorithm works in EasyTPP

    main

    EasyTPP uses an EventSampler to implement the thinning procedure for sampling event sequences. This is particularly useful for intensity-based models.

    The draw_next_time_one_step method performs the following steps for every history:

    1. Proposal Rate Calculation: Evaluates total intensity at num_samples_boundary points and multiplies the maximum by over_sample_rate to establish a proposal-rate bound.
    2. Proposal Generation: Draws num_exp exponential increments at that rate and applies torch.cumsum to create accumulated proposal times.
    3. Intensity Evaluation: Evaluates the model's marked intensities at these proposal times and sums over marks.
    4. Acceptance/Rejection: Draws uniform random variables and accepts the first proposal that satisfies the thinning criterion.
    5. Fallback: If no proposal is accepted, it returns dtime_max for that sample. The final prediction for the next interval is the equal-weighted average of the num_sample draws.

    Once a time is sampled, marked intensities at that time are normalized over event types to predict the mark.

  5. Configure sequence padding and masking

    main

    EasyTPP performs event tokenize which includes padding and masking sequences.

    Padding

    By default, time_seqs, time_delta_seqs, and type_seqs are padded to the longest sequence in the batch.

    • To use a fixed length, set padding_strategy to max_length and provide max_len in the configuration.
    • The default padding value for time_pad and type_pad is num_event_types (assuming event indices start at 0).

    Masking

    After padding, two masks are generated:

    1. batch_non_pad_mask: Indicates the position of non-padding elements in the sequence.
    2. attention_mask: Used for attention calculations, ensuring an event can only attend to its past events.

    Each batch contains: time_seqs, time_delta_seqs, type_seqs, batch_non_pad_mask, and attention_mask.

  6. Understand the EasyTPP model validation process

    main

    EasyTPP validates model implementations by training on a dataset for a specified maximum number of epochs. The validation process follows these steps:

    1. Training: The model is trained for a set number of epochs.
    2. Selection: The 'best' model is selected based on its performance on the validation set.
    3. Evaluation: Final performance metrics are reported using the test set.

    Key metrics used for evaluation include:

    • Loglike: Log-likelihood.
    • RMSE: Root Mean Square Error.
    • Acc: Accuracy.

  7. Prepare Data for EasyTPP

    main

    EasyTPP supports data in legacy pickle representation or JSON files. You can also use datasets hosted by the EasyTPP organization on Hugging Face.

    All supported data formats must provide the following fields:

    • time_since_start
    • time_since_last_event
    • type_event
    • dim_process

    The Preprocess module uses these fields to generate padded model batches. For custom data loading, you can implement a pseudo dataloader using the TPPDataLoader pipeline.

  8. Mathematical Foundation of Log-likelihood in EasyTPP

    main

    The log-likelihood of the model parameters is calculated as the sum of the log-intensities of the events that occurred at their specific times, minus the integral of the total intensities over the observation interval $[0, T]$:

    $$\sum_{t_i}\log \lambda_{k_i}(t_i) - \int_0^T \lambda(t) dt$$

    • Event Term: $\sum_{t_i}\log \lambda_{k_i}(t_i)$
    • Non-event Term: $-\int_0^T \lambda(t) dt$

  9. Create an experiment configuration with YAML

    main

    EasyTPP uses YAML files to define experiment pipelines. A configuration typically includes a pipeline_config_id, a data section defining dataset paths and specifications (like num_event_types and pad_token_id), and specific task configurations (e.g., NHP_train) that inherit from a base_config and define trainer_config and model_config parameters.

    pipeline_config_id: runner_config

data:
  taxi:
    data_format: json
    train_dir: easytpp/taxi
    valid_dir: easytpp/taxi
    test_dir: easytpp/taxi
    data_specs:
      num_event_types: 10
      pad_token_id: 10
      padding_side: right
      truncation_side: right

NHP_train:
  base_config:
    stage: train
    backend: torch
    dataset_id: taxi
    runner_id: std_tpp
    model_id: NHP
    base_dir: ./checkpoints/
  trainer_config:
    gpu: -1
    batch_size: 256
    max_epoch: 1
    shuffle: true
    metrics: [acc, rmse]
  model_config:
    hidden_size: 64
    loss_integral_num_sample_per_step: 20
    thinning:
      num_sample: 1
      num_exp: 500
      over_sample_rate: 5
      num_samples_boundary: 5
      dtime_max: 5
      patience_counter: 5
      num_step_gen: 1

  12. Explore EasyTPP via Colab Tutorials

    main

    The following Google Colab notebooks are available for interactive learning:

    • Tutorial 1: Dataset in EasyTPP: Understanding dataset structures.
    • Tutorial 2: Tensorboard in EasyTPP: Visualizing training with Tensorboard.
    • Tutorial 3: Training and Evaluation of TPPs: Standard training workflows.
    • Deep dive: log-space statistics in IntensityFree: Detailed look at log-space statistics.
    • Deep dive: multi-step event generation and intensity-free thinning: Advanced event generation techniques.