TimesFM (Time Series Foundation Model)

You can estimate required RAM using the following formula:

RAM ≈ model_weights + 0.5 GB + (0.2 MB × num_series × context_length / 1000)

Key Constants for TimesFM 2.5 (200M):

• model_weights: ~800 MB
• context_length: Your max_context value
• num_series: Number of time series in your batch

GPU VRAM considerations:

• Model weights: ~800 MB
• KV cache + activations: ~200–500 MB (scales with context)
• Batch buffers: ~100 MB per 100 series at context=1024.

TimesFM 2.5 introduces several improvements over version 2.0:

• Model Size: Uses 200M parameters (down from 500M).
• Context Length: Supports up to 16k context length (up from 2048).
• Quantile Forecasting: Supports continuous quantile forecast up to a 1k horizon via an optional 30M quantile head.
• Covariate Support: Re-introduced support for covariates through XReg.
• Simplified Configuration: Removes the frequency indicator and adds new forecasting flags.

TimesFM requires input data to be provided as a list of 1-D numpy arrays. Each array in the list represents a single univariate time series.

Key Requirements:

• Dimensionality: Each array must be 1-dimensional.
• Data Type: Use np.float32 or np.float64.
• Variable Lengths: Series within the same batch can have different lengths.
• NaN Handling:
  • Leading NaNs are automatically stripped.
  • Internal NaNs are automatically linearly interpolated.
  • Trailing NaNs are NOT handled and must be removed manually before passing data to the model.

import numpy as np

inputs = [
    np.array([1.0, 2.0, 3.0, 4.0, 5.0]),       # Series 1
    np.array([10.0, 20.0, 15.0, 25.0]),          # Series 2 (different length)
    np.array([100.0, 110.0, 105.0, 115.0, 120.0, 130.0]),  # Series 3
]

The PEFT pipeline supports four distinct fine-tuning strategies. You can toggle between these by modifying the finetune.sh script or using the corresponding command-line flags:

• Full Fine-Tuning: Adjusts all model parameters during training.
• LoRA (Low-Rank Adaptation): A memory-efficient method that fine-tunes a small number of parameters by decomposing weight matrices into low-rank matrices. Use the --use-lora flag.
• DoRA (Directional LoRA): An extension of LoRA that decomposes pre-trained weights into magnitude and direction components, using LoRA for directional adaptation. Use the --use-dora flag.
• Linear Probing: Fine-tunes only the residual blocks and the embedding layer, leaving other parameters unchanged. Use the --use-linear-probing flag.

When fine-tuning TimesFM 2.5, keep these two concepts in mind:

• No External Normalisation: TimesFM 2.5 uses internal instance normalisation (RevIN). Do not normalise your data externally; feed raw values directly to the model.
• Random Window Sampling: Training examples are created by slicing random (context, horizon) windows from input series. This improves data efficiency compared to using fixed windows.

To reproduce the global temperature anomaly forecast report, follow these steps to install dependencies and run the provided scripts. This example uses a 36-month historical dataset to forecast a 12-month horizon.

Prerequisites

Ensure you have uv installed for dependency management.

Steps

1. Install the required dependencies including the PyTorch backend.
2. Navigate to the example directory.
3. Execute the one-click runner script.

# Install dependencies
uv pip install "timesfm[torch]" matplotlib pandas numpy

# Run the complete example
cd scientific-skills/timesfm-forecasting/examples/global-temperature
./run_example.sh

Contributors are requested to use yapf for code formatting following the Google style. Use the following configuration in your .style.yapf file:

[style]
based_on_style = google
# Add your custom style rules here
indent_width = 2
spaces_before_comment = 2

Before submitting a PR, run the following command on all affected files:

yapf --in-place --recursive <filename>

GPU Acceleration

On Ampere+ GPUs (e.g., A100, RTX 3090+), always set: torch.set_float32_matmul_precision("high")

Batch Size Guidelines

Memory-Constrained Processing

If you encounter Out-of-Memory (OOM) errors, process your input series in chunks:

CHUNK = 50
results = []
for i in range(0, len(inputs), CHUNK):
    p, q = model.forecast(horizon=H, inputs=inputs[i:i+CHUNK])
    results.append((p, q))

The forecast_with_covariates function (used for external regressors) requires JAX and jaxlib. If you installed the base PyTorch version, you must manually install these dependencies to avoid errors when calling the method, as it relies on the xreg_lib module.

pip install jax jaxlib

Install the timesfm package using uv (recommended for speed) or pip. You must specify the backend you intend to use.

For PyTorch backend:

uv pip install timesfm[torch]
# or
pip install timesfm[torch]

For JAX/Flax backend (optimized for TPU/GPU):

uv pip install timesfm[flax]

Note on PyTorch installation: After installing the package, ensure you install the correct torch version for your hardware (CUDA, CPU, or Apple Silicon/MPS).

You can install TimesFM directly from PyPI using specific extras depending on your preferred backend and Python version.

• For the PyTorch version (requires Python >= 3.11):

  pip install timesfm[torch]

• For the PAX version (requires Python 3.10.x):

  pip install timesfm[pax]