tsfresh Documentation

tsfresh is a Python package designed for systematic feature engineering from time-series and other sequential data (data ordered by an independent variable like time or wavelength). It automates the process of calculating hundreds of characteristics—such as maximum, minimum, average, or number of peaks—that would otherwise require manual calculation.

Key capabilities:

• Automated Extraction: Automatically calculates and returns a wide range of features from sequential data.
• Integration: Fully compatible with pandas and scikit-learn, allowing for easy integration into existing data science workflows.
• Use Cases: Extracted features can be used for describing time-series dynamics, clustering, and training machine learning models for classification or regression tasks.

tsfresh (Time Series Feature extraction based on scalable hypothesis tests) is a Python package designed for systematic time-series feature extraction. It automates the process of extracting hundreds of features from time-series data—ranging from basic statistics (mean, max) to complex signal processing and nonlinear dynamics characteristics.

Key capabilities include:

• Automatic Extraction: Generates hundreds of features to describe time-series characteristics.
• Built-in Filtering: Uses a statistically robust filtering procedure based on hypothesis testing to remove irrelevant or redundant features for specific regression or classification tasks.
• Versatility: Works with sampled data, event sequences, and even spatial variation sequences (SVS) from images.
• Compatibility: Integrates with pandas, numpy, and scikit-learn.

The tsfresh feature filtering process consists of three distinct phases:

1. Feature extraction: Time series are characterized using feature mappings from tsfresh.feature_extraction.feature_calculators to derive aggregated features.
2. Feature significance testing: Each feature vector is evaluated for its significance in predicting the target using tests found in tsfresh.feature_selection.significance_tests. This produces a vector of p-values.
3. Multiple test procedure: The p-values are evaluated using the Benjamini-Yekutieli procedure (via the statsmodel package) to decide which features to keep.

tsfresh supports parallelization for feature extraction, feature selection, and rolling window calculations. You can control the level of parallelization using the n_jobs and chunksize parameters.

• n_jobs: Specifies the number of worker processes. It defaults to the number of processors on the current system. Setting n_jobs=0 disables parallelization, which is useful for profiling.
• chunksize: Defines the number of chunks (where one chunk is a singular time series for one id and one kind) submitted as a single task to a worker process. Setting this to None uses heuristics to find an optimal size. Optimizing chunksize via benchmarks is crucial for performance.

Note: If you are working with data that exceeds memory capacity, refer to the large data documentation instead of just tuning parallelization.

tsfresh provides three scikit-learn compatible transformers to integrate time series feature extraction and selection into machine learning pipelines:

1. FeatureAugmenter: Extracts features from time series.
2. FeatureSelector: Performs feature selection algorithms.
3. RelevantFeatureAugmenter: Combines both extraction and filtering in a single step. This is the preferred method to avoid unnecessary feature calculations.

These transformers allow you to include feature engineering as a pre-processing step in a sklearn.pipeline.Pipeline, enabling cross-validation of the entire sequence.

To quickly try out tsfresh or integrate it into your workflow using a containerized environment, you can pull the official Docker image.

docker pull nbraun/tsfresh

For Windows users, it is recommended to use Anaconda. Note that tsfresh uses multiprocessing, which can sometimes cause issues on Windows. Follow these steps to set up a dedicated environment:

1. Open the Anaconda Prompt.
2. Create and configure the environment using the commands below.

conda create -n ENV_NAME python=VERSION
conda install -n ENV_NAME pip requests numpy pandas scipy statsmodels patsy scikit-learn tqdm
activate ENV_NAME
pip install tsfresh

A Stacked DataFrame uses a single column for all values and a separate column to identify the type of time series. This format is advantageous because timestamps for different series do not need to align.

Example Structure:

Usage: Set both column_kind and column_value to the names of your respective columns. You can omit column_value and let tsfresh attempt to deduce it.

column_id="id", column_sort="time", column_kind="kind", column_value="value"

To set up a local development environment for tsfresh, install the package in editable mode with testing dependencies and initialize pre-commit hooks. Using the -e flag ensures that changes to the code are immediately reflected in your test runs.

cd /path/to/tsfresh
pip install -e ".[testing]"
pre-commit install