tsdownsample

When using tsdownsample, ensure your data meets the following requirements:

1. x-data must be non-strictly monotonic increasing (sorted).
2. x-data must not contain any NaN values.

You can install the tsdownsample package using pip. This package provides extremely fast time series downsampling written in Rust with PyO3 bindings.

pip install tsdownsample

The tsdownsample module provides high-performance time series downsampling algorithms implemented in Rust. It is organized into submodules based on the algorithm used (minmax, m4, lttb, minmaxlttb) and the execution mode (sequential or parallel).

Each algorithm submodule contains further submodules for sequential and parallel execution. Functions within these submodules allow for downsampling either with or without explicit x-axis data (assuming constant sampling rates) and can handle NaN values in specific versions.

import tsdownsample

# Accessing algorithms
minmax = tsdownsample.minmax
m4 = tsdownsample.m4
lttb = tsdownsample.lttb
minmaxlttb = tsdownsample.minmaxlttb

The MinMaxLTTB algorithm uses the following parameters:

• n_out: The target number of points to return in the downsampled output.
• minmax_ratio: A threshold used to decide if a preliminary min-max aggregation step is needed. If input_length / n_out > minmax_ratio, the algorithm first performs a min-max aggregation to reduce the data size before applying the standard LTTB (Largest Triangle Three Buckets) algorithm. This improves performance on very large datasets.

All downsamplers support a parallel keyword argument to enable multi-threading.

To control the maximum number of threads used by the Rust backend, set the TSDOWNSAMPLE_MAX_THREADS environment variable.

import os
os.environ["TSDOWNSAMPLE_MAX_THREADS"] = "4"

To use tsdownsample, instantiate one of the available downsampler classes and call the .downsample() method. The method returns an array of indices (ndarray[uint64]) which you can then use to slice your original data.

from tsdownsample import MinMaxLTTBDownsampler
import numpy as np

# Create a time series
y = np.random.randn(10_000_000)
x = np.arange(len(y))

# Downsample to 1000 points (assuming constant sampling rate)
s_ds = MinMaxLTTBDownsampler().downsample(y, n_out=1000)

# Select downsampled data
downsampled_y = y[s_ds]

# Downsample to 1000 points using the (possible irregularly spaced) x-data
s_ds = MinMaxLTTBDownsampler().downsample(x, y, n_out=1000)

# Select downsampled data
downsampled_x = x[s_ds]
downsampled_y = y[s_ds]

Each downsampling algorithm is implemented as a class with a downsample method.

Method Signature: downsample([x], y, n_out, **kwargs) -> ndarray[uint64]

Arguments:

• x (optional): The x-axis data (must be non-strictly monotonic increasing and contain no NaNs).
• y (mandatory): The y-axis data.
• n_out (mandatory keyword): The desired number of output values. Note that if there are gaps in the time series, fewer than n_out indices may be returned.
• **kwargs (optional): Algorithm-specific arguments (see algorithm table).

Returns: A ndarray[uint64] of indices used to index the original data.

The downsample_rs crate uses a global thread pool (POOL) powered by rayon for parallel execution. You can control the number of threads used by the pool by setting the TSDOWNSAMPLE_MAX_THREADS environment variable.

If this variable is not set, the library defaults to the number of available parallel threads on the system. If parallelism cannot be determined, it defaults to 1 thread.

export TSDOWNSAMPLE_MAX_THREADS=4

The library provides specific classes to handle NaN values in the y data. You can either omit NaNs or return the index of the first NaN encountered in a bin.

Note: NaNs are not supported for x-data.

The following classes are available for downsampling:

For downsamplers that need special handling for NaN values (typically only applicable to floating-point y data), subclass AbstractRustNaNDownsampler.

This class uses the prefix downsample_nan to look up specialized functions in the Rust module. If the y data is not a floating-point type, it automatically strips the _nan suffix to fall back to standard downsampling functions, as non-float types cannot contain NaN values.