tick: Statistical Learning for Time-Dependent Modelling

repository·master·Indexed 19 days ago

https://github.com/x-datainitiative/tick

A Python 3 library for statistical learning specializing in time-dependent modeling and point processes, such as Hawkes processes. Version 0.8.0.2 provides a robust optimization toolbox with batch and stochastic solvers, generalized linear models (linear, logistic, and Poisson regression), survival analysis, and robust inference tools. It includes specialized modules for proximal operators, dataset simulation, and preprocessing, with performance optimizations using Intel® Math Kernel Library (MKL).

Tokens
25.6K
Snippets
76
Records
118
Agent score
66%

What's inside tick

  1. Overview of the tick library

    master
    tick is a Python 3 machine learning library focused on statistical learning for time-dependent systems, such as point processes. It provides tools for generalized linear models, generic optimization (including solvers and proximal operators for weight penalization), and various dataset simulation tools.
  2. Overview of tick capabilities

    master

    tick is a Python 3 machine learning library focused on statistical learning for time-dependent systems, such as point processes.

    Core features include:

    • Optimization Module: Provides model computational classes, solvers, and proximal operators for regularization.
    • Generalized Linear Models: Tools for performing linear, logistic, or Poisson regression.
    • Point Processes: Tools to simulate and infer Hawkes processes with various kernel assumptions (e.g., exponential, sum of exponential, linear combination of basis kernels, or sparse interactions).
    • Performance: Optimized using Intel® Math Kernel Library (MKL) for efficient execution on Intel® Xeon Phi™ and Intel® Xeon™ processors.
  3. Use robust losses for regression and classification

    master

    When your data contains outliers, you can use specific robust loss models instead of standard least-squares.

    Regression Models (Continuous Labels)

    • ModelHuber: Uses Huber loss, which is quadratic for small errors and linear for large errors. Tune the transition point using the threshold argument ($\delta$).
    • ModelEpsilonInsensitive: Uses epsilon-insensitive loss, which ignores errors smaller than a certain threshold. Tune the threshold using the threshold argument ($\epsilon$).
    • ModelAbsoluteRegression: Uses L1 loss ($|y' - y|$), which is inherently more robust to outliers than L2 loss.
    • ModelLinRegWithIntercepts: The underlying model used by RobustLinearRegression.

    Classification Models (Binary Labels)

    • ModelModifiedHuber: A modified Huber loss designed for robust binary classification (where $y \in {-1, 1}$), making it less sensitive to outliers in the feature space.
  4. Combine multiple proximal operators with ProxMulti

    master

    The ProxMulti class allows you to combine multiple proximal operators. It applies each operator passed to it sequentially, one after the other. This is useful for applying different types of penalization to different parts of a vector.

    from tick.prox import ProxMulti, ProxL1, ProxTV
    
    # Example: combining total-variation and L1 penalization
    prox = ProxMulti([ProxTV(), ProxL1(strength=1e-2)])
  5. Inference and simulation of Hawkes processes with `tick.hawkes`

    master

    The tick.hawkes module provides tools for working with Hawkes processes, including learners for parameter inference, kernels for defining process dynamics, and simulators for generating point process data.

    Key Components:

    Learners (Inference): Use these classes to estimate Hawkes process parameters from data. Examples include:

    • HawkesExpKern, HawkesSumExpKern, HawkesBasisKernels: Kernel-based learners.
    • HawkesEM, HawkesADM4: Optimization-based learners.
    • HawkesCumulantMatching: Moment-based learner.

    Kernels: Define the functional form of the excitation in a Hawkes process:

    • HawkesKernelExp, HawkesKernelSumExp, HawkesKernelPowerLaw, HawkesKernelTimeFunc.

    Simulation: Generate synthetic point process data using:

    • SimuPoissonProcess, SimuInhomogeneousPoisson.
    • SimuHawkes, SimuHawkesExpKernels, SimuHawkesSumExpKernels, SimuHawkesMulti.

    Models: Represent the statistical model for likelihood calculation or least squares fitting:

    • ModelHawkesExpKernLogLik, ModelHawkesExpKernLeastSq, ModelHawkesSumExpKernLogLik, ModelHawkesSumExpKernLeastSq.
  6. Link Python attributes to C++ setters

    master

    To ensure a C++ object stays in sync with a Python object, you can link a Python attribute to a specific C++ setter method. This is common when a Python class wraps a C++ object used for heavy computations.

    To implement this, you must define two things in your class:

    1. _cpp_obj_name: The name of the attribute holding the C++ object (e.g., '_a').
    2. cpp_setter in _attrinfos: The name of the C++ method to call when the Python attribute is modified.

    When the Python attribute is updated, the specified C++ method is automatically called with the new value.

    from tick.base.build.base import A0 as _A
    from tick.base import Base
    
    class A(Base):
        _attrinfos = {
            'cpp_int': {'cpp_setter': 'set_cpp_int'},
            '_a' : {'writable' : False}
        }
        _cpp_obj_name = "_a"
    
        def __init__(self):
            self._a = _A()
            self.cpp_int = 0
    
    a = A()
    a.cpp_int = -4  # Automatically calls a._a.set_cpp_int(-4)
  7. Split serialization into load and save methods

    master

    If an archive member requires complex initialization during the loading process, you can split the single serialize method into two distinct methods: save (for serialization) and load (for deserialization).

    template <class Archive>
    void save(Archive & ar) const {
      ar(x);
      ar(y);
      ar(z.get_foo());
    }
    
    template <class Archive>
    void load(Archive & ar) {
      ar(x);
      ar(y);
    
      float temp = 0.0f;
      ar(temp);
    
      z = Z(temp);
    }
  8. Survival analysis with `tick.survival`

    master

    The tick.survival module provides tools for survival analysis (time-to-event analysis).

    Inference:

    Functions for estimating survival parameters:

    • CoxRegression
    • nelson_aalen
    • kaplan_meier

    Models:

    Classes representing survival models:

    • ModelCoxRegPartialLik (Cox Proportional Hazards)
    • ModelSCCS (Structured Continuous Change)

    Simulation:

    • SimuCoxReg: Simulates data following a Cox regression model.
  9. Understand proximal operators in tick.prox

    master

    The tick.prox module provides proximal operators used for model fitting with penalization. In optimization problems of the form $\min_w f(w) + g(w)$, $f$ is the goodness-of-fit term and $g$ is the penalization function.

    The proximal operator of a convex function $g$ at point $w$ with regularization parameter $t$ is defined as: $$\text{prox}{g}(w, t) = \text{argmin}{w'} \Big{ \frac 12 | w - w' |_2^2 + t g(w') \Big$$

    Many prox classes support a range parameter, which allows applying the regularization only to a specific subset of the entries in the weight vector $w$. This is useful for models where certain components (like intercepts) should not be penalized.

  10. Understand deterministic vs stochastic solvers in tick

    master

    The tick.solver toolbox categorizes solvers into two main types based on how they handle data:

    • Deterministic solvers: These perform a full pass over the entire dataset at each iteration.
    • Stochastic solvers: These perform multiple iterations (controlled by the epoch_size parameter) within each main iteration, making them more efficient for large datasets.