DOCSEARCHbyAlgolia
Submit Docs

gym-mtsim Documentation

repository·main·Indexed 19 days ago

https://github.com/aminhp/gym-mtsim

A trading simulator and OpenAI Gym/Gymnasium environment for MetaTrader 5. It enables simulation of Forex, Stocks, Crypto, and Futures trading, detailed backtesting via the MtSimulator engine, and training of reinforcement learning agents using MtEnv. The library integrates with stable-baselines3 and supports both hedge and unhedge trading strategies across various asset classes.

Tokens
3.6K
Snippets
8
Records
12
Agent score
68%

What's inside gym-mtsim

  1. Use MtEnv for Reinforcement Learning

    main

    The MtEnv class is an OpenAI Gym environment built on top of MtSimulator. It is designed for training RL agents.

    Action Space Structure

    Because stable-baselines does not support Dict or 2D Box spaces, the action space is flattened into a 1D vector of size count(trading_symbols) * (symbol_max_orders + 2).

    For each symbol, the vector contains:

    1. Probabilities of closing existing orders (up to symbol_max_orders).
    2. A value for holding or creating a new order.
    3. The volume of the new order (positive for Buy, negative for Sell).

    Note on Probabilities: The environment assumes probability values follow the logit function. The step method applies the expit function to map these values back to the $[0, 1]$ range.

    Observation Space

    Each observation includes:

    • balance, equity, margin.
    • features: A window of signal_features of length window_size.
    • orders: A 3D array indexed by [symbol_index, order_number, [entry_price, volume, profit]].

  2. Install gym-mtsim

    main

    You can install gym-mtsim using PIP or by cloning the repository for development. Additionally, stable-baselines3 is required if you intend to run the provided examples.

    Prerequisites

    1. Install MetaTrader 5: Download and install the software from the official website.
    2. Open a Demo Account: Ensure you have a demo account active in MetaTrader 5 to facilitate data usage.

    Installation Commands

    # Via PIP
pip install gym-mtsim

# From Repository (Editable mode)
git clone https://github.com/AminHP/gym-mtsim
cd gym-mtsim
pip install -e .

# Or via direct ZIP archive
pip install --upgrade --no-deps --force-reinstall https://github.com/AminHP/gym-mtsim/archive/main.zip

# Install stable-baselines3 for examples
pip install stable-baselines3

  3. Use MtEnv for Reinforcement Learning with Gymnasium

    main

    The MtEnv class provides a Gymnasium-compatible environment for training RL agents.

    Default Environments

    You can quickly create default environments using gym.make():

    • forex-hedge-v0 / forex-unhedge-v0
    • stocks-hedge-v0 / stocks-unhedge-v0
    • crypto-hedge-v0 / crypto-unhedge-v0
    • mixed-hedge-v0 / mixed-unhedge-v0

    Custom Environments

    To create a custom environment, pass an initialized MtSimulator instance to MtEnv. Key parameters include:

    • original_simulator: An instance of MtSimulator.
    • trading_symbols: A list of symbols to trade.
    • window_size: The lookback window for features.
    • hold_threshold: Threshold to hold an order.
    • close_threshold: Threshold to close an order.
    • fee: A function (e.g., lambda symbol: ...) that returns the fee for a given symbol.
    • symbol_max_orders: Maximum number of orders allowed per symbol.
    • multiprocessing_processes: Number of processes for parallel environments.
    import gymnasium as gym
import gym_mtsim

# Default environment
env = gym.make('forex-hedge-v0')

# Custom environment
from gym_mtsim import MtEnv, MtSimulator, FOREX_DATA_PATH
import numpy as np

sim = MtSimulator(unit='USD', balance=10000., leverage=100., hedge=True, symbols_filename=FOREX_DATA_PATH)

env = MtEnv(
    original_simulator=sim,
    trading_symbols=['GBPCAD', 'EURUSD'],
    window_size=10,
    hold_threshold=0.5,
    close_threshold=0.5,
    fee=lambda symbol: 0.0002,
    symbol_max_orders=2
)

  4. Use MtSimulator for manual trading simulation

    main

    The MtSimulator class allows you to simulate MetaTrader 5 trading logic manually. You can initialize it with custom parameters like unit, balance, leverage, and stop_out_level. You can load existing symbol data using load_symbols or download new data using download_data with a specified time_range and timeframe.

    To simulate time passing, use sim.tick(timedelta). You can place orders using sim.create_order and close them using sim.close_order. The current state of the simulator (balance, equity, margin, etc.) can be retrieved via sim.get_state().

    import pytz
from datetime import datetime, timedelta
from gym_mtsim import MtSimulator, OrderType, Timeframe, FOREX_DATA_PATH

sim = MtSimulator(
    unit='USD',
    balance=10000.,
    leverage=100.,
    stop_out_level=0.2,
    hedge=False,
)

# Load or download data
if not sim.load_symbols(FOREX_DATA_PATH):
    sim.download_data(
        symbols=['EURUSD', 'GBPCAD'],
        time_range=(
            datetime(2021, 5, 5, tzinfo=pytz.UTC),
            datetime(2021, 9, 5, tzinfo=pytz.UTC)
        ),
        timeframe=Timeframe.D1
    )
    sim.save_symbols(FOREX_DATA_PATH)

# Place an order
sim.current_time = datetime(2021, 8, 30, 0, 17, 52, tzinfo=pytz.UTC)
order1 = sim.create_order(
    order_type=OrderType.Buy,
    symbol='GBPCAD',
    volume=1.,
    fee=0.0003,
)

# Advance time
sim.tick(timedelta(days=2))

# Get state
state = sim.get_state()
print(state['balance'])

# Close order
sim.close_order(order1)

  5. Configure MtEnv properties

    main

    When initializing or configuring an MtEnv, you can control several parameters:

    • trading_symbols: List of symbols to trade.
    • time_points: The timeline for the simulator (defaults to the index of the first symbol's DataFrame).
    • hold_threshold / close_threshold: Probability thresholds for order management.
    • fee: A constant or a callable that returns a fee for a given symbol.
    • symbol_max_orders: Max open positions per symbol (for hedge trading).
    • window_size: The length of the feature window for observations.
    • multiprocessing_processes: Max processes for parallel execution.

  6. Create a gym-mtsim environment

    main

    You can create trading environments using gym.make() by specifying the environment name. The available environments follow a naming convention for asset classes (forex, stocks, crypto, mixed) and trading strategies (hedge, unhedge).

    Available environment names include:

    • forex-hedge-v0 / forex-unhedge-v0
    • stocks-hedge-v0 / stocks-unhedge-v0
    • crypto-hedge-v0 / crypto-unhedge-v0
    • mixed-hedge-v0 / mixed-unhedge-v0
    • stocks-unhedge-v0 (and other variations)
    import gymnasium as gym
import gym_mtsim

env_name = 'stocks-hedge-v0'
env = gym.make(env_name)

  7. Train an agent using stable-baselines3 and gym-mtsim

    main

    You can integrate gym-mtsim with stable-baselines3 for reinforcement learning. The following pattern demonstrates initializing an environment, training an A2C model, and running a prediction loop.

    import gymnasium as gym
from gym_mtsim import MtEnv
from stable_baselines3 import A2C
import numpy as np
import torch

env_name = 'forex-hedge-v0'

# Set seeds for reproducibility
seed = 2024

env = gym.make(env_name)
model = A2C('MultiInputPolicy', env, verbose=0)
model.learn(total_timesteps=1000)

observation, info = env.reset(seed=seed)

while True:
    action, _states = model.predict(observation)
    observation, reward, terminated, truncated, info = env.step(action)
    done = terminated or truncated

    if done:
        break

env.unwrapped.render('advanced_figure', time_format='%Y-%m-%d')

  8. Render MtEnv visualizations

    main

    The MtEnv supports two rendering modes to visualize trading activity:

    1. simple_figure: A basic plot where each symbol has a unique color. Green/red triangles represent Buy/Sell actions, gray triangles indicate errors, and black vertical bars indicate close actions.
    2. advanced_figure: An interactive plot. You can click symbol names to toggle visibility and hover over markers for details. The size of the triangles corresponds to the trade volume.

    Note: When using advanced_figure, you can pass a time_format string (e.g., time_format='%Y-%m-%d').

    # Simple visualization
env.render('simple_figure')

# Interactive visualization
env.render('advanced_figure', time_format='%Y-%m-%d')

  9. Use MtSimulator for core trading simulation

    main

    The MtSimulator class is the engine that simulates MetaTrader's core mechanics. It can be used independently of the Gym environment for backtesting or analysis.

    Key Properties

    • balance: Money before open positions.
    • equity: Total money including open positions.
    • margin: Required margin for open positions.
    • free_margin: Available funds for new positions.
    • margin_level: Ratio of equity to margin.
    • stop_out_level: Threshold where the broker automatically closes unprofitable positions.
    • hedge: Boolean indicating if hedging is enabled.
    • orders / closed_orders: Lists of active and completed orders.

    Key Methods

    • download_data: Downloads symbol data from MetaTrader (Windows only; requires MetaTrader5 Python package).
    • save_symbols / load_symbols: Persist/load symbol data to/from files.
    • tick: Advances the simulation by a delta time and updates all properties.
    • create_order: Places a Buy or Sell order.
    • close_order: Closes an existing order.
    • get_state: Returns a system state similar to the MetaTrader Toolbox window.

  10. Understand the Order data class

    main

    The Order data class contains all information regarding a specific trade. Key properties include:

    • id: Unique identifier for tracking.
    • type: Enum specifying Buy or Sell.
    • symbol: The traded asset.
    • volume: The size of the order (must be a multiple of volume_step between volume_min and volume_max).
    • fee: Represents the bid/ask spread (since MetaTrader does not have a native 'fee' concept).
    • entry_time / exit_time: Timestamps for order placement and closure.
    • entry_price / exit_price: The close price at entry and exit.
    • profit: Current or realized profit.
    • margin: Required margin for the order.
    • closed: Boolean indicating if the order is finished.

  11. Train gym-mtsim agents with Stable-Baselines3

    main

    To train reinforcement learning agents on gym-mtsim environments, use stable-baselines3 algorithms like A2C or PPO. Since these environments often involve complex observation spaces, it is recommended to use the MultiInputPolicy.

    Note: If you want to use the built-in progress bar in model.learn(), ensure you have installed the extra packages: pip install stable-baselines3[extra].

    from stable_baselines3 import A2C, PPO
import gym
import gym_mtsim

env_name = 'stocks-hedge-v0'
env = gym.make(env_name)

# Use MultiInputPolicy for complex observation spaces
model = PPO('MultiInputPolicy', env, verbose=1)
model.learn(total_timesteps=10000)

  12. Implement a ProgressBarCallback for model.learn()

    main

    If you are not using the stable-baselines3[extra] installation, you can implement a custom ProgressBarCallback using tqdm to track training progress during the model.learn() call.

    from tqdm import tqdm
from stable_baselines3.common.callbacks import BaseCallback

class ProgressBarCallback(BaseCallback):
    def __init__(self, check_freq: int, verbose: int = 1):
        super().__init__(verbose)
        self.check_freq = check_freq

    def _on_training_start(self) -> None:
        self.progress_bar = tqdm(total=self.model._total_timesteps, desc="model.learn()")

    def _on_step(self) -> bool:
        if self.n_calls % self.check_freq == 0:
            self.progress_bar.update(self.check_freq)
        return True

    def _on_training_end(self) -> None:
        self.progress_bar.close()