DOCSEARCHbyAlgolia
Submit Docs

Greykite Documentation

repository·master·Indexed 23 days ago

https://github.com/linkedin/greykite

A flexible and fast forecasting and anomaly detection library featuring the Silverkite algorithm. Greykite provides tools for time series forecasting via the forecast_pipeline, hierarchical forecast reconciliation using ReconcileAdditiveForecasts, and structural changepoint detection. It includes various templates such as SilverkiteTemplate, ProphetTemplate, and AutoArimaTemplate, as well as a Benchmark class for comparing model performance using Rolling Window Cross-Validation.

Tokens
33.4K
Snippets
52
Records
119
Agent score
84%

What's inside Greykite

  1. Overview of the Greykite forecasting library

    master

    Greykite is a forecasting and anomaly detection library developed by LinkedIn. Its primary algorithm, Silverkite, is designed for univariate forecasting and is capable of capturing:

    • Time-varying growth
    • Seasonality
    • Autocorrelation
    • Holidays
    • Regressors

    Silverkite provides pre-tuned templates (parameter configurations) optimized for different forecast frequencies, horizons, and data patterns. Beyond Silverkite, Greykite provides interfaces for other forecasting models including Prophet (by Facebook) and Auto-ARIMA (via pmdarima).

    The library supports the full forecasting lifecycle, including exploratory data analysis (EDA), end-to-end forecasting, model tuning, and benchmarking.

  2. Compare Greykite forecasting models

    master

    Greykite provides three primary forecasting models: Silverkite, Prophet, and ARIMA. Choosing between them depends on your requirements for speed, accuracy, interpretability, and ease of use.

    Model Comparison Summary

    FeatureSILVERKITEPROPHETARIMA
    Speedfastslowfast
    Forecast accuracy (default)decentdecentdecent
    Forecast accuracy (customized)very goodgoodgood
    Interpretabilitygoodgooddecent
    Ease of usegoodgoodvery good
    API Stylesklearnsimilar to sklearnsimilar to sklearn

  3. Overview of Greykite Model Template Categories

    master

    Model templates in Greykite are categorized by the underlying algorithm and the level of abstraction they provide:

    • Silverkite (High-level): Uses SimpleSilverkiteTemplate. Includes "AUTO" and strings starting with "SILVERKITE". Tailored for various horizons and frequencies.
    • Silverkite (Low-level): Uses SilverkiteTemplate (template name "SK"). Intended for advanced users to tune lower-level parameters.
    • Prophet: Uses ProphetTemplate (template name "PROPHET").
    • ARIMA: Uses AutoArimaTemplate (template name "AUTO_ARIMA").
    • Lag-based: Uses LagBasedTemplate (template name "LAG_BASED"). Useful for simple baselines like week-over-week.
    • Multistage: Uses MultistageForecastTemplate. Fits multiple models sequentially to residuals, where the final prediction is the sum of the models.

  4. Overview of Greykite notable components

    master

    Greykite provides several specialized components for forecasting and analysis:

    • ModelSummary(): Provides R-like summaries for scikit-learn and statsmodels regression models.
    • ChangepointDetector(): Performs changepoint detection based on adaptive lasso and includes visualization capabilities.
    • SimpleSilverkiteForecast(): A simplified interface to the Silverkite algorithm providing forecast_simple and predict methods.
    • SilverkiteForecast(): The low-level interface to the Silverkite algorithm providing forecast and predict methods.
    • ReconcileAdditiveForecasts(): Adjusts a set of forecasts to satisfy inter-forecast additivity constraints.
    • GreykiteDetector(): A simple interface for optimizing anomaly detection performance based on Greykite forecasts.

  5. What is Silverkite?

    master

    Silverkite is a forecasting algorithm developed by LinkedIn. It works by generating basis functions for various components such as growth, seasonality, and holidays. These features, along with any provided regressors, are used to fit the time series.

    Key characteristics:

    • Flexible: Supports various growth types, interactions, and fitting algorithms.
    • Interpretable: Uses additive fitting algorithms by default, allowing you to identify the specific contribution of each component.
    • Fast: Designed to run significantly faster than Bayesian alternatives.

  6. Overview of Greykite components

    master

    Greykite provides several specialized components for forecasting and analysis:

    • ModelSummary(): Provides R-like summaries for scikit-learn and statsmodels regression models.
    • ChangepointDetector(): Detects changepoints using adaptive lasso, including visualization capabilities.
    • SimpleSilverkiteForecast(): A simplified Silverkite interface with forecast_simple and predict methods.
    • SilverkiteForecast(): The low-level interface to the Silverkite algorithm with forecast and predict methods.
    • ReconcileAdditiveForecasts(): Adjusts multiple forecasts to satisfy inter-forecast additivity constraints.
    • GreykiteDetector(): A simple interface for optimizing anomaly detection performance based on Greykite forecasts.

  7. Configure custom seasonalities using add_seasonality_dict

    master

    The add_seasonality_dict parameter in the Prophet seasonality configuration allows you to add custom seasonal components or override built-in ones.

    Each entry in the dictionary uses a component name (e.g., 'monthly') as the key. The value is a dictionary containing:

    • period: The periodicity of the component.
    • fourier_order: The number of Fourier terms.
    • prior_scale: (Optional) The strength of this specific seasonality.
    • mode: (Optional) 'additive' or 'multiplicative' specifically for this component.

    To override a built-in seasonality like weekly, you must first set weekly_seasonality=False in the main config.

    seasonality = dict(
    weekly_seasonality=[False],
    add_seasonality_dict=[
        {
            'weekly': {
                'period': 7,
                'fourier_order': 1.0,
                'prior_scale': 5.0,
                'mode': "multiplicative"
            }
        }
    ]
)

  8. Greykite Anomaly Detection (AD)

    master

    Greykite AD is an extension of the forecasting library designed to monitor metrics with minimal effort. It improves upon standard Silverkite confidence intervals by automatically tuning them based on expected alert rates and/or available anomaly labels.

    Users can define robust objective functions, constraints, and parameter spaces to optimize detection. For example, you can target a specific minimum recall (e.g., 80%) while maximizing precision, or specify a minimum error level to ensure detected anomalies are business-relevant rather than just statistically significant.

  9. Tune seasonality flexibility with Fourier order

    master

    The Fourier order k acts as a tuning knob for model flexibility. A model with Fourier order k includes 2k seasonality terms.

    Guidelines for choosing Fourier order:

    • Higher values: More flexible, but risk overfitting to past data.
    • Lower values: Less flexible, more robust.
    • Constraint: The order k should satisfy 2k <= n_levels + 1, where n_levels is the number of possible values in a cycle (e.g., n_levels=7 for weekly seasonality).
    • Typical values: Generally <= 4, up to 12 for daily seasonality, and up to 15 for yearly seasonality.
    • Best Practice: Plot your time series to check the strength of seasonality. For example, monthly seasonality often has a weak effect and is typically limited to a Fourier order of 2.

  10. How hyperparameter_override updates model_components

    master

    The hyperparameter_override attribute works by updating or adding to the parameters defined in model_components.

    • If a key in an override dictionary matches a key in the original grid, it replaces that parameter for that specific search space.
    • If a key is new, it is added to the parameters for that search space.

    This results in a list of grids that is passed to sklearn.model_selection.RandomizedSearchCV. Greykite automatically converts single values into lists (e.g., 5 becomes [5]) to ensure compatibility with the underlying scikit-learn search mechanism.

    # Original grid defined by the other model_components attributes.
original_grid = {"estimator__param": 5}

# Override options.
hyperparameter_override = [
    {},
    {"estimator__param": [10]},
    {"estimator__param1": ["a", "b", "c"]},
    {"estimator__param2": [1.0, 2.0]},
]

# The resulting search space passed to RandomizedSearchCV:
hyperparameter_grid = [
    {"estimator__param": [5]},
    {"estimator__param": [10]},
    {"estimator__param": [5], "estimator__param1": ["a", "b", "c"]},
    {"estimator__param": [5], "estimator__param2": [1.0, 2.0]},
]

  11. Reconcile forecasts using ReconcileAdditiveForecasts

    master

    Use greykite.algo.reconcile.convex.reconcile_forecasts.ReconcileAdditiveForecasts to ensure that a set of forecasts satisfy inter-forecast additivity constraints (e.g., the sum of regional forecasts must equal the total company revenue forecast).

    This method is a post-hoc reconciliation technique. It takes existing base forecasts and applies a linear transformation $T$ to produce adjusted forecasts $F_{adj}$ that satisfy linear constraints $C F_{adj} = 0$.

    Key Workflow:

    1. Generate base forecasts using any algorithm.
    2. Define a constraint matrix $C$ representing your additive relationships.
    3. Use ReconcileAdditiveForecasts to compute the optimal transformation matrix $T$.
    4. Apply $T$ to your base forecasts to get consistent results.

    For a practical implementation guide, see the tutorial: /gallery/quickstart/04_postprocessing/0100_reconcile_forecasts.

  12. Configure Rolling Window Cross-Validation (CV)

    master

    Greykite uses Rolling Window Cross-Validation instead of standard K-fold CV to respect temporal dependencies in time-series data (preventing the use of future data to predict the past).

    In this approach:

    • A series of $K$ test sets (BM-folds) is created.
    • For each test set, observations prior to the set are used for training.
    • Within each training set, internal CV folds are created to optimize parameters.
    • The number of datapoints in every test and validation set equals the forecast horizon.

    You can define this behavior using the greykite.sklearn.cross_validation.RollingTimeSeriesSplit class.