DOCSEARCHbyAlgolia
Submit Docs

pytimetk Documentation

repository·master·Indexed 21 days ago

https://github.com/business-science/pytimetk

A high-performance time-series toolkit for Python (v2.5.0.9000) providing a unified API for pandas, Polars, and cuDF. It features tools for visualization, time-aware aggregations, anomaly detection, regime modeling, and feature engineering. Includes a Beta FeatureStore for caching transforms, Ray-powered parallel execution, and native Polars support for optimized performance in EWM and padding operations.

Tokens
5.5K
Snippets
13
Records
26
Agent score
76%

What's inside pytimetk

  1. Overview of pytimetk workflows and APIs

    master

    pytimetk provides a unified API for time-series analysis that works with both pandas and Polars (and optionally NVIDIA cudf for GPU acceleration).

    Core Workflows

    WorkflowKey API MethodsDescription
    Visualization & Diagnosticsplot_timeseries, plot_stl_diagnostics, plot_time_series_boxplot, theme_plotly_timetkInteractive Plotly charts and STL faceting.
    Time-aware Aggregationssummarize_by_time, apply_by_time, pad_by_timeResampling, rolling up, and padding missing rows.
    Feature Engineeringaugment_timeseries_signature, augment_rolling, augment_wavelet, feature_storeCalendar signatures, rolling windows, and wavelets.
    Anomaly Workflowsanomalize, plot_anomalies, plot_anomalies_decomp, plot_anomalies_cleanedDetect, diagnose, and visualize anomalies.
    Finance & Regimesaugment_regime_detection, augment_macdHMM-based regime detection and financial indicators.
    Polars-native.tk accessor on pl.DataFrameUse engine="polars" to stay within the Polars ecosystem.
    Production (Beta)FeatureStore, GPU accelerationCaching transforms and RAPIDS/GPU support.

  2. Use tidy-style column selectors in pytimetk

    master

    pytimetk provides reusable helpers for expressive column selection, allowing you to target columns based on patterns rather than exact names. These selectors work across both pandas and polars dataframes.

    Available selector helpers include:

    • contains(pattern): Selects columns containing a specific string.
    • starts_with(pattern): Selects columns starting with a specific string.
    • ends_with(pattern): Selects columns ending with a specific string.
    • matches(regex): Selects columns matching a regular expression.
    • resolve_column_selection(...): A utility to resolve these selectors into actual column names.

    These selectors are integrated into many high-level APIs, such as plotting functions and feature engineering operations, to provide a more flexible way to handle time-series data.

  3. Planned pytimetk visualization functions

    master

    The pytimetk library is working towards feature parity with the R timetk package by implementing a suite of specialized time-series visualization helpers. These functions are designed to be Pythonic, interoperable with pandas, and return plotly Figure objects for interactive use in notebooks and dashboards.

    Available/Implemented functions:

    • plot_acf_diagnostics()
    • plot_stl_diagnostics()
    • plot_seasonal_diagnostics()
    • plot_time_series_boxplot()
    • plot_time_series_regression()

    Planned functions:

    • plot_time_series_cv_plan()

  4. Understand Polars-native vs. Pandas fallback behavior

    master

    pytimetk is optimized to keep data in its native engine (Polars, cuDF, or pandas) as long as possible to avoid expensive serialization and memory copies.

    Polars Native Paths

    Several features now support native Polars execution:

    • pad_by_time and future_frame.
    • EWM (Exponential Weighted Moving) via _augment_ewm_polars (supports most decay parameters/functions natively).
    • augment_rolling_apply and augment_expanding_apply (keeps data in Arrow buffers, only converting to pandas when a user-supplied Python callable is executed).
    • The scalar branch of apply_by_time.

    Known Fallbacks

    • Wide Format: The wide_format=True branch in apply_by_time currently falls back to pandas to preserve MultiIndex behavior. This is a documented limitation.
    • Unsupported Parameters: If an EWM function or decay parameter is not supported natively in Polars, the toolkit will automatically fall back to pandas.

  5. Supported frequency specification formats in pytimetk

    master

    As part of the datetime alias and duration migration, pytimetk supports three distinct ways to specify time frequencies and durations in its APIs (such as pad_by_time, future_frame, and rolling/expanding utilities):

    1. Human-friendly durations: Natural language strings like "3 weeks" or "2 months".
    2. Current pandas aliases: Standard pandas frequency strings such as "ME" (Month End), "QE" (Quarter End), "YE" (Year End), "SME" (Semi-Month End), etc.
    3. Legacy pandas aliases: Older pandas frequency strings like "M", "Q", "Y", "SM", "BM", "BQ", "BY", and "CBM". These are internally remapped to the new aliases to ensure backward compatibility without breaking functionality.

  6. Usage patterns for pytimetk plotting functions

    master

    When using the new visualization functions in pytimetk, expect the following API patterns:

    1. Return Type: Functions return a plotly.graph_objects.Figure object. This allows you to embed plots in Jupyter notebooks, dashboards, or save them as static artifacts.
    2. Rendering Control: Functions accept a show=False keyword argument. Use this to defer rendering if you need to modify the figure object (e.g., updating layout or traces) before displaying it.
    3. Data Input: Functions follow the standard pytimetk pattern: passing in a DataFrame and using tidy column selection (specifying column names for the time and value components).

  8. How to control parallel execution in pytimetk

    master

    Many pytimetk functions (such as future_frame, ts_features, and various rolling/expanding augmentation helpers) now use Ray to dispatch grouped work across multiple threads/processes for improved performance.

    To control this behavior:

    • Enable Parallelism: Ensure threads is set to a value other than 1. The toolkit will attempt to use Ray to distribute the workload.
    • Disable Parallelism: If you encounter issues with Ray (e.g., worker startup errors or resource constraints) or if you are running in an environment where Ray is not supported, set threads=1. This forces the functions to use a sequential fallback path.

  9. Automatic sorting in time-series feature engineering

    master

    Most feature engineering and diagnostic functions in pytimetk automatically handle chronological sorting to ensure mathematical correctness. You do not need to manually sort your data before calling these functions, as they use internal helpers like sort_dataframe to sort by group + date and then restore the original index to maintain alignment with your input.

    Functions that automatically enforce sorting include:

    • Regime Detection: augment_regime_detection (sorts before computing log-returns and fitting HMMs).
    • Decompositions: stl_diagnostics and plot_stl_diagnostics (sorts before fitting STL).
    • Rolling/Expanding Windows: augment_rolling, augment_rolling_apply, and augment_expanding (sorts per group/date to ensure sequential window calculations).
    • Shift-based Features: augment_diffs, augment_lags, and augment_leads (sorts before applying differences or shifts, then restores original order).
    • Future Generation: future_frame (sorts timestamps to correctly infer frequency and extrapolate dates).

  10. How `pytimetk` handles data sorting internally

    master

    To maintain data integrity and ensure that time-series operations (like rolling windows or HMM training) are mathematically sound, pytimetk follows a consistent internal pattern:

    1. Sort: The data is sorted by group (if applicable) and date using the sort_dataframe helper.
    2. Compute: The time-series operation (e.g., rolling mean, lag, or regime detection) is performed on the ordered data.
    3. Restore: The resulting DataFrame is re-sorted back to the original input index/order using a synthetic row_id or index restoration. This ensures that the output DataFrame can be joined back to the original data without alignment issues.

  11. Quickstart with pytimetk

    master

    This example demonstrates the core workflow: loading a dataset, performing time-aware aggregations using the Polars engine, visualizing the results, filling time gaps, and detecting anomalies.

    import numpy as np
import pandas as pd
import pytimetk as tk
from pytimetk.utils.selection import contains

# Load sample data
sales = tk.load_dataset("bike_sales_sample", parse_dates=["order_date"])

# 1. Summaries in one line (Polars engine for speed)
monthly = (
    sales.groupby("category_1")
    .summarize_by_time(
        date_column="order_date",
        value_column="total_price",
        freq="MS",
        agg_func=["sum", "mean"],
        engine="polars",
    )
)

# 2. Visualize straight from Polars/pandas
monthly.plot_timeseries(
    date_column="order_date",
    value_column=contains("sum"),
    color_column="category_1",
    title="Revenue by Category",
    plotly_dropdown=True,
)

# 3. Fill gaps + detect anomalies
hourly = (
    sales.groupby(["category_1", "order_date"], as_index=False)
    .agg(total_price=("total_price", "sum"))
    .groupby("category_1")
    .pad_by_time(date_column="order_date", freq="1H", fillna=0)
)

anomalies = (
    hourly.groupby("category_1")
    .anomalize("order_date", "total_price")
    .plot_anomalies(date_column="order_date", plotly_dropdown=True)
)
    import numpy as np
import pandas as pd
import pytimetk as tk
from pytimetk.utils.selection import contains

sales = tk.load_dataset("bike_sales_sample", parse_dates=["order_date"])

# 1. Summaries in one line (Polars engine for speed)
monthly = (
    sales.groupby("category_1")
    .summarize_by_time(
        date_column="order_date",
        value_column="total_price",
        freq="MS",
        agg_func=["sum", "mean"],
        engine="polars",
    )
)

# 2. Visualize straight from Polars/pandas
monthly.plot_timeseries(
    date_column="order_date",
    value_column=contains("sum"),
    color_column="category_1",
    title="Revenue by Category",
    plotly_dropdown=True,
)

# 3. Fill gaps + detect anomalies
hourly = (
    sales.groupby(["category_1", "order_date"], as_index=False)
    .agg(total_price=("total_price", "sum"))
    .groupby("category_1")
    .pad_by_time(date_column="order_date", freq="1H", fillna=0)
)

anomalies = (
    hourly.groupby("category_1")
    .anomalize("order_date", "total_price")
    .plot_anomalies(date_column="order_date", plotly_dropdown=True)
)