TradeMaster RL Platform

TradeMaster is an open-source platform for quantitative trading (QT) using reinforcement learning (RL). It provides a complete pipeline for designing, implementing, evaluating, and deploying RL-based trading algorithms. The architecture consists of six core modules:

1. Multi-modality market data: Supports various financial assets at multiple granularities.
2. Data preprocessing pipeline: A complete workflow for preparing market data.
3. Market simulators: High-fidelity, data-driven simulators designed for mainstream QT tasks.
4. RL algorithms: Efficient implementations of over 13 novel RL-based trading algorithms.
5. Evaluation toolkits: Systematic evaluation tools covering 6 axes and 17 measures.
6. Interdisciplinary interfaces: Various interfaces designed to accommodate users from different backgrounds.

TradeMaster is an open-source reinforcement learning (RL) platform designed for quantitative trading (QT). It provides a full pipeline for designing, implementing, evaluating, and deploying RL-based trading algorithms. The platform is composed of six key modules:

1. Multi-modality market data: Supports various financial assets at multiple granularities.
2. Data preprocessing pipeline: A complete workflow for preparing financial data.
3. Market simulators: High-fidelity, data-driven simulators for mainstream QT tasks.
4. RL algorithms: Efficient implementations of over 13 novel RL-based trading algorithms.
5. Evaluation toolkits: Systematic evaluation tools using 6 axes and 17 measures.
6. Interdisciplinary interfaces: Various interfaces designed for users from different backgrounds.

The repository is organized into several functional directories:

• configs/: Configuration files.
• data/: Datasets categorized by algorithmic_trading, high_frequency_trading, order_execution, and portfolio_management.
• deploy/: Backend service and client scripts (backend_service.py, backend_client.py).
• installation/: Setup guides for Docker and requirements.
• tools/: Specialized tools for algorithmic_trading, data_preprocessor, high_frequency_trading, market_dynamics_labeling, missing_value_imputation, order_execution, and portfolio_management.
• tradmaster/: The core library containing agents, datasets, environments, evaluation, imputation, losses, nets, preprocessor, optimizers, pretrained, trainers, transition, and utils.
• unit_testing/: Test suites.

Use the Market Dynamics Modeling module to label raw datasets with interpretable and controllable dynamics. This allows you to use these dynamics as meta-information for targeted policy evaluation.

Workflow:

1. Configure Parameters: Edit the configuration file located at configs/market_dynamics_modeling/market_dynamics_modeling.py. For detailed information on the modeling algorithm and parameters, refer to the TradeMaster_Sandbox_whitepaper.pdf.
2. Run Labeling: Execute the labeling script to prepare the dataset.
3. Tune Results: Review the experiment logs and generated visualizations (metrics_of_each_dynamics.png and slice_and_merge_model_{dynamic_number}dynamics_minlength{min_length_limit}_{labeling_method}_labeling_{tic}.png) to adjust hyperparameters until the dynamics meet your requirements.

python tools/market_dynamics_labeling/run.py

TradeMaster requires Python 3.9+. It is recommended to use Miniconda. Use the following commands to update your base conda installation, create a new environment named TradeMaster, and activate it.

conda update -n base -c defaults conda
conda create --name TradeMaster python=3.9
conda activate TradeMaster

High Frequency Trading (HFT) in TradeMaster involves buying or selling a pre-selected financial instrument periodically within seconds, accounting for order execution. The HFT_DDQN approach utilizes a double Q network and a decayed supervised regulator (generated from a real Q-table based on future price information) to optimize profit margins.

You can implement this using the provided Jupyter Notebook or the standalone training script.

After preparing your labeled dataset, you can perform evaluation on a specific market dynamic by running the trainer with the dynamics_test task. This will generate reports and visualization results (such as Radar plots).

Use the --task_name dynamics_test argument and specify the target dynamic index using --test_dynamic.

python tools/algorithmic_trading/train.py --task_name dynamics_test --test_dynamic 0

The State Augmented Reinforcement Learning (SARL) method for portfolio management uses asset prices (internal features) and Financial News (external information) to construct an augmented state $s$. The asset movement signal $\delta$ is used for state augmentation, allowing the policy network to generate management strategies from the augmented state.

You can implement the training and testing process for SARL using the provided notebook or Python script within the TradeMaster framework.

PRUDEX-Compass provides high-level benchmarking using 6 axes and 16 measures. You can generate a compass visualization by running the create_compass.py script with a JSON data file.

JSON Data Format Requirements: Each entry in the JSON list must define:

• color: One of ["magenta", "green", "blue", "orange", "cyan", "brown"].
• label: A string describing the entry.
• inner_level: An integer attribute value between 1 and 100.
• outer_level: A boolean value (true or false).

After building the image, you can verify the installation by running an interactive container and executing a training script. Use docker run -it to enter the container's bash shell, then run the tools/algorithmic_trading/train.py script to start an experiment.

docker images
docker run -it --name trademaster trademaster:1.0.0 /bin/bash
# Inside the container:
python tools/algorithmic_trading/train.py

Run the algorithmic trading training script to confirm that the installation was successful.

python tools/algorithmic_trading/train.py