Precipitation Nowcasting

The VarFlow Python wrapper requires a two-step installation process: first, compiling the underlying C++ core using cmake, and second, installing the Python package using setup.py.

1. Compile the C++ Core

Create a build directory and run cmake. Depending on your operating system, use the appropriate commands below.

Windows

Use the Visual Studio generator:

mkdir build
cmake -G "Visual Studio 14 2015 Win64" ^
-DCMAKE_BUILD_TYPE=Release ^
-DCMAKE_CONFIGURATION_TYPES="Release" ^
..

Linux

mkdir build
cd build
cmake ..

Handling OpenCV Errors

If cmake fails to find OpenCV, manually specify the path using the -DOpenCV_DIR flag:

mkdir build
cd build
cmake -DOpenCV_DIR=/usr/local/software/opencv/share/OpenCV ..
make

2. Install the Python Package

Once the build is successful, validate the installation by running the example script located at varflow/varflow.py. If it runs without error, install the package in development mode:

python setup.py develop

This entrypoint script configures and executes a training and testing loop for the TrajGRU model architecture using a balanced Weighted_mse_mae loss function. It integrates an Encoder and a Forecaster into an EF (Encoder-Forecaster) wrapper. The training process uses the Adam optimizer and a MultiStepLR learning rate scheduler.

import torch
from nowcasting.config import cfg
from nowcasting.models.forecaster import Forecaster
from nowcasting.models.encoder import Encoder
from nowcasting.models.model import EF
from nowcasting.models.loss import Weighted_mse_mae
from nowcasting.models.trajGRU import TrajGRU
from nowcasting.train_and_test import train_and_test
from experiments.net_params import encoder_params, forecaster_params

# Configuration
batch_size = cfg.GLOBAL.BATCH_SZIE
max_iterations = 100000
test_iteration_interval = 1000
test_and_save_checkpoint_iterations = 1000
LR = 1e-4

# Model Setup
criterion = Weighted_mse_mae().to(cfg.GLOBAL.DEVICE)
encoder = Encoder(encoder_params[0], encoder_params[1]).to(cfg.GLOBAL.DEVICE)
forecaster = Forecaster(forecaster_params[0], forecaster_params[1]).to(cfg.GLOBAL.DEVICE)
encoder_forecaster = EF(encoder, forecaster).to(cfg.GLOBAL.DEVICE)

# Optimization
optimizer = torch.optim.Adam(encoder_forecaster.parameters(), lr=LR)
mult_step_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones=[30000, 60000], gamma=0.1)

# Execution
train_and_test(
    encoder_forecaster, 
    optimizer, 
    criterion, 
    mult_step_scheduler, 
    batch_size, 
    max_iterations, 
    test_iteration_interval, 
    test_and_save_checkpoint_iterations, 
    folder_name
)

To execute a Conv2D-based precipitation nowcasting training experiment, use the train_and_test function. This function orchestrates the training loop, evaluation, and checkpoint saving. It requires a model, an optimizer, a loss criterion, a learning rate scheduler, and several hyperparameter/interval settings.

Key parameters for train_and_test:

• model: The neural network instance (e.g., a Predictor configured with conv2d_params).
• optimizer: The optimization algorithm (e.g., torch.optim.Adam).
• criterion: The loss function (e.g., Weighted_mse_mae).
• exp_lr_scheduler: The learning rate scheduler (e.g., torch.optim.lr_scheduler.StepLR).
• batch_size: Number of samples per training batch.
• max_iterations: Total number of training iterations.
• test_iteration_interval: Frequency of evaluation iterations.
• test_and_save_checkpoint_iterations: Frequency of evaluation and checkpoint saving.
• folder_name: The directory name used for organizing experiment outputs.

from nowcasting.train_and_test import train_and_test

# Example setup
train_and_test(
    model, 
    optimizer, 
    criterion, 
    exp_lr_scheduler, 
    batch_size=4, 
    max_iterations=80000, 
    test_iteration_interval=10000, 
    test_and_save_checkpoint_iterations=10000, 
    folder_name='conv2d'
)