TODS (Automated Time-series Outlier Detection System)

TODS follows a modular machine learning pipeline consisting of 6 core modules:

• Data Processing: Handles tabular data operations including dataset loading, filtering, validation, binarization, and timestamp transformation.
• Timeseries Processing: Provides time-series-specific preprocessing such as seasonality/trend decomposition, transformation, scaling, and smoothing.
• Feature Analysis: Performs exhaustive feature extraction across three domains: Time domain, Frequency domain, and Latent factor models (includes 30 methods like statistical methods, spectral transformations, and matrix factorization).
• Detection Algorithms: Implements various approaches for the three outlier scenarios, including traditional (e.g., IForest, Autoregression), heuristic (e.g., HotSax, Matrix Profile), deep learning (e.g., RNN-LSTM, GAN, VAE), and ensemble methods.
• Reinforcement Module: Allows users to improve models using human expertise. Currently supports rule-based filtering to transform domain knowledge into rule filters.

You can build an automated outlier detection pipeline by composing PrimitiveStep objects within a Pipeline object from the d3m library. Each step in the pipeline uses a specific primitive (e.g., data transformation, scaling, or detection algorithms) and connects its output to the input of the next step using data_reference strings (e.g., steps.N.produce).

Common steps in a TODS pipeline include:

• dataset_to_dataframe: Converts a dataset to a DataFrame.
• column_parser: Parses columns into semantic types.
• extract_columns_by_semantic_types: Filters columns based on types like Attribute or TrueTarget.
• axiswise_scaler: Scales time-series data.
• pyod_ae: An Autoencoder-based detection algorithm.
• construct_predictions: Formats the final output.

from d3m import index
from d3m.metadata.base import ArgumentType
from d3m.metadata.pipeline import Pipeline, PrimitiveStep

# Creating pipeline
pipeline_description = Pipeline()
pipeline_description.add_input(name='inputs')

# Step 0: dataset_to_dataframe
step_0 = PrimitiveStep(primitive=index.get_primitive('d3m.primitives.data_transformation.dataset_to_dataframe.Common'))
step_0.add_argument(name='inputs', argument_type=ArgumentType.CONTAINER, data_reference='inputs.0')
step_0.add_output('produce')
pipeline_description.add_step(step_0)

# ... add subsequent steps connecting via data_reference ...

# Final Output
pipeline_description.add_output(name='output predictions', data_reference='steps.6.produce')

# Export to JSON
data = pipeline_description.to_json()
with open('example_pipeline.json', 'w') as f:
    f.write(data)

TODS allows for automated searching of the best pipeline for a given dataset using a Searcher.

Workflow:

1. Generate a problem_description using generate_problem(dataset, metric).
2. Initialize a backend (e.g., axolotl.backend.simple.SimpleRunner).
3. Initialize BruteForceSearch with the problem description and backend.
4. Run search.search_fit(input_data=[dataset], time_limit=seconds) to find the optimal pipeline.
5. Use search.evaluate(best_pipeline) to get scores.

from tods import generate_dataset, generate_problem
from tods.searcher import BruteForceSearch
from axolotl.backend.simple import SimpleRunner
import pandas as pd

# Setup
df = pd.read_csv('yahoo_sub_5.csv')
dataset = generate_dataset(df, target_index=6)
problem_description = generate_problem(dataset, metric='F1_MACRO')

# Search
backend = SimpleRunner(random_seed=0)
search = BruteForceSearch(problem_description=problem_description, backend=backend)

best_runtime, best_pipeline_result = search.search_fit(input_data=[dataset], time_limit=30)

# Results
best_pipeline = best_runtime.pipeline
best_output = best_pipeline_result.output
print(f'Best Pipeline ID: {best_pipeline.id}')

To execute a pipeline defined in a JSON file, use generate_dataset to prepare your data and evaluate_pipeline to run the pipeline and get results.

1. Prepare Dataset: Use generate_dataset(df, target_index) where target_index is the column index of the ground truth.
2. Load Pipeline: Use load_pipeline(pipeline_path).
3. Evaluate: Use evaluate_pipeline(dataset, pipeline, metric) where metric can be 'F1' or 'F1_MACRO'.

from tods import generate_dataset, load_pipeline, evaluate_pipeline
import pandas as pd

# Load data
df = pd.read_csv('your_data.csv')
target_index = 6

# Prepare
dataset = generate_dataset(df, target_index)
pipeline = load_pipeline('autoencoder_pipeline.json')

# Run
pipeline_result = evaluate_pipeline(dataset, pipeline, metric='F1_MACRO')
print(pipeline_result)

from tods import generate_dataset, load_pipeline, evaluate_pipeline
import pandas as pd

# Read data and generate dataset
df = pd.read_csv(table_path)
dataset = generate_dataset(df, target_index)

# Load the default pipeline
pipeline = load_pipeline(pipeline_path)

# Run the pipeline
pipeline_result = evaluate_pipeline(dataset, pipeline, metric)
print(pipeline_result)

If you want to find the best pipeline for your dataset automatically, use the BruteForceSearch class from tods.searcher.

Workflow:

1. Generate Problem: Create a problem_description using generate_problem(dataset, metric).
2. Setup Backend: Initialize a SimpleRunner (e.g., backend = SimpleRunner(random_seed=0)).
3. Initialize Searcher: search = BruteForceSearch(problem_description=problem_description, backend=backend).
4. Search: Call search.search_fit(input_data=[dataset], time_limit=seconds).
5. Retrieve Results: The best pipeline is found in best_runtime.pipeline.

from tods import generate_dataset, generate_problem
from tods.searcher import BruteForceSearch
from axolotl.backend.simple import SimpleRunner

# 1. Prepare data and problem
dataset = generate_dataset(df, target_index=6)
problem_description = generate_problem(dataset, metric='F1_MACRO')

# 2. Setup
backend = SimpleRunner(random_seed=0)
search = BruteForceSearch(problem_description=problem_description, backend=backend)

# 3. Search
# time_limit is in seconds
best_runtime, best_pipeline_result = search.search_fit(input_data=[dataset], time_limit=30)

# 4. Results
best_pipeline = best_runtime.pipeline
print(f"Best Pipeline ID: {best_pipeline.id}")

# Read data and generate dataset and problem
df = pd.read_csv(table_path)
dataset = generate_dataset(df, target_index=target_index)
problem_description = generate_problem(dataset, metric)

# Start backend
backend = SimpleRunner(random_seed=0)

# Start search algorithm
search = BruteForceSearch(problem_description=problem_description,
                          backend=backend)

# Find the best pipeline
best_runtime, best_pipeline_result = search.search_fit(input_data=[dataset], time_limit=time_limit)
best_pipeline = best_runtime.pipeline
best_output = best_pipeline_result.output

TODS uses the D3M framework to define pipelines as a sequence of PrimitiveStep objects. A pipeline typically involves data processing, feature extraction, an algorithm, and a prediction construction step.

Steps in a typical pipeline:

1. Data Loading: d3m.primitives.tods.data_processing.dataset_to_dataframe
2. Parsing: d3m.primitives.tods.data_processing.column_parser
3. Feature Selection: d3m.primitives.tods.data_processing.extract_columns_by_semantic_types (using types like Attribute or TrueTarget)
4. Feature Analysis: e.g., d3m.primitives.tods.feature_analysis.statistical_maximum
5. Algorithm: e.g., d3m.primitives.tods.detection_algorithm.pyod_ae (AutoEncoder)
6. Predictions: d3m.primitives.tods.data_processing.construct_predictions

Once built, the pipeline can be exported to JSON using pipeline_description.to_json().

from d3m import index
from d3m.metadata.pipeline import Pipeline, PrimitiveStep
from d3m.metadata.base import ArgumentType

# Creating pipeline
pipeline_description = Pipeline()
pipeline_description.add_input(name='inputs')

# Example Step: dataset_to_dataframe
step_0 = PrimitiveStep(primitive=index.get_primitive('d3m.primitives.tods.data_processing.dataset_to_dataframe'))
step_0.add_argument(name='inputs', argument_type=ArgumentType.CONTAINER, data_reference='inputs.0')
step_0.add_output('produce')
pipeline_description.add_step(step_0)

# ... add other steps ...

# Output to json
data = pipeline_description.to_json()
with open('autoencoder_pipeline.json', 'w') as f:
    f.write(data)