Merlion Documentation

Merlion models are categorized into several functional groups:

• Anomaly Detection (merlion.models.anomaly): Models designed to detect outliers or unusual patterns.
• Forecast Models (merlion.models.forecast): Models designed for time series forecasting.
• Forecast-based Anomaly Detection (merlion.models.anomaly.forecast_based): A subset of anomaly detection models that utilize forecasting models at their core.
• Ensembles (merlion.models.ensemble): Support for combining multiple models.
• AutoML (merlion.models.automl): Automated machine learning capabilities.

The merlion.transform package is organized into several functional areas:

Base Primitives

• transform.factory: Provides the TransformFactory for creating transforms by name.
• transform.base: Contains base classes for transforms.
• transform.sequence: Utilities for sequential transformations.

Resampling

• transform.resample: Tools for changing the frequency or resolution of time series.
• transform.moving_average: Implements moving average smoothing.

Normalization

• transform.normalize: Tools for scaling and normalizing data (e.g., Rescale).
• transform.bound: Tools for bounding data within specific ranges.

Miscellaneous

• transform.anomalize: Specialized transformation utilities.

The merlion.evaluate sub-package provides utilities and metrics to quantitatively evaluate the performance of time series models. It is organized into three main modules based on the task type:

1. merlion.evaluate.base: Contains base evaluation utilities.
2. merlion.evaluate.anomaly: Contains metrics specifically for anomaly detection tasks.
3. merlion.evaluate.forecast: Contains metrics specifically for forecasting tasks.

Merlion provides several anomaly detection models that operate by forecasting future values and identifying deviations between the forecast and the actual observed values. These models are located in the merlion.models.anomaly.forecast_based module.

Available model implementations include:

• arima (AutoRegressive Integrated Moving Average)
• sarima (Seasonal AutoRegressive Integrated Moving Average)
• ets (Exponential Smoothing)
• prophet (Facebook Prophet implementation)
• mses (Multiple Seasonal Exponential Smoothing)

Merlion provides several Automated Machine Learning (AutoML) models designed to automatically select and tune time series models. The available AutoML model implementations include:

• AutoETS: Automated Exponential Smoothing models.
• AutoProphet: Automated Prophet models.
• AutoSARIMA: Automated Seasonal Autoregressive Integrated Moving Average models.

These models are part of the merlion.models.automl module.

The merlion.spark package allows you to use distributed computing to train and run inference on multiple time series in parallel.

You can use the PySpark API in two ways:

1. Directly: Invoke the Spark apps spark_apps/anomaly.py or spark_apps/forecast.py from the command line using python or spark-submit.
2. Kubernetes: Use the provided Dockerfile to serve a Spark application on a Kubernetes cluster with spark-on-k8s.

To see the expected arguments for the standalone apps, run:

python spark_apps/anomaly.py -h
# or
python spark_apps/forecast.py -h

The merlion.post_process package provides tools to refine the output of anomaly detection models using simple rules. You can use this package to:

• Calibrate sequences to follow a standard normal distribution using merlion.post_process.calibrate.
• Sparsify sequences based on a specific threshold using merlion.post_process.threshold.
• Compose multiple post-processing steps into a single pipeline using merlion.post_process.sequence.
• Instantiate post-processors via merlion.post_process.factory.

Use the third tab in the dashboard to train forecasting models. Follow these steps:

1. Select Dataset:
   • Use a single dataset with a train/test split fraction.
   • Or, select "Separate train/test files" to use a dedicated test dataset.
2. Configure Columns:
   • Target column (Required): The variable you wish to forecast.
   • Additional features (Optional): For multivariate forecasting.
   • Exogenous variables (Optional): Variables whose values are known a priori.
3. Select Algorithm: Choose a forecasting algorithm (e.g., Arima, AutoETS) and modify hyperparameters as needed.

Output: Trained models are saved in ~/merlion/models/<algorithm_name>. The dashboard will display forecasting results on the test set and performance metrics.

Merlion provides a web-based GUI dashboard for quickly experimenting with models on custom datasets. To use it, install Merlion with the dashboard optional dependency and run the module via the command line. The dashboard will be available at http://localhost:8050.

pip install salesforce-merlion[dashboard]
python -m merlion.dashboard

The ts_datasets package provides standardized data loaders for multiple time series datasets. To use it without manually specifying the root directory of every dataset, you must install it in editable mode from the cloned repository.

pip install -e Merlion/ts_datasets/