FinRL-X Quantitative Trading Infrastructure

repository·master·Indexed 25 days ago

https://github.com/ai4finance-foundation/finrl-trading

An AI-native, modular quantitative trading infrastructure featuring a weight-centric architecture. FinRL-X unifies data processing (FMP, Yahoo Finance, WRDS), strategy composition (DRL, Mean-Variance, Adaptive Rotation), backtesting via the bt engine, and brokerage execution through Alpaca. It supports a full workflow from stock selection and portfolio allocation to live paper trading, with deployment options via CLI, Python API, or Docker Compose.

Tokens
26.3K
Snippets
55
Records
149
Agent score
86%

What's inside FinRL-X

  1. Overview of FinRL-X Architecture

    master

    FinRL-X is a modular quantitative trading infrastructure designed around a weight-centric architecture. The core principle is that the target portfolio weight vector serves as the sole interface contract between strategy logic and execution. This allows for modular swapping of components (e.g., replacing an Equal Weight allocator with a DRL allocator) without changing the downstream pipeline.

    The architecture is organized into four layers:

    1. Data: Handles market data via FMP, Yahoo Finance, WRDS, and LLM sentiment preprocessing, using SQLite for caching.
    2. Strategy: Generates weight-centric signals through stock selection, portfolio allocation, timing adjustment, and risk overlay.
    3. Backtest: An offline evaluation engine powered by bt that supports multi-benchmark comparison and transaction costs.
    4. Execution: Manages live or paper trading via Alpaca multi-account integration with pre-trade risk checks.
  2. Configure Min Variance weight allocation

    master

    The min_variance method requires historical price data to calculate volatility. Depending on your data source, you must provide the data in one of two formats within the data_dict.

    Option 1: Using Fundamental Data (Quarterly)

    If using fundamental data, the system relies on the adj_close_q column. Use a shorter lookback_periods (e.g., 8 quarters).

    Option 2: Using Price Data (Daily)

    If using daily price data, provide a prices key in the data_dict. Use a longer lookback_periods (e.g., 252 days).

    Parameters

    • lookback_periods: The number of historical periods to use for volatility calculation.
    # Option 1: Using fundamental data (adj_close_q)
    data_dict = {
        'fundamentals': fundamentals_df
    }
    
    result = strategy.generate_weights(
        data_dict,
        prediction_mode='single',
        weight_method='min_variance',
        lookback_periods=8
    )
    
    # Option 2: Using daily price data
    data_dict = {
        'fundamentals': fundamentals_df,
        'prices': prices_df  # Must contain ['date', 'tic', 'close']
    }
    
    result = strategy.generate_weights(
        data_dict,
        prediction_mode='single',
        weight_method='min_variance',
        lookback_periods=252
    )
  3. Run the FinRL Full Workflow tutorial

    master

    The FinRL_Full_Workflow.ipynb Jupyter notebook is the recommended starting point. It covers the complete quantitative trading workflow: data acquisition (S&P 500 components, fundamentals, historical prices), ML strategy implementation (Random Forest), professional backtesting with benchmarks (VOO, QQQ), and live trading execution via Alpaca Paper Trading.

    jupyter notebook examples/FinRL_Full_Workflow.ipynb
  4. Setup the ML Stock Selection environment

    master

    Before running the pipeline, configure your environment and install dependencies.

    1. Environment Variables: Create a .env file in the project root with your FMP API key: FMP_API_KEY=your_fmp_api_key_here

    2. Dependencies:

    python3 -m venv venv
    source venv/bin/activate
    pip install pandas numpy scikit-learn requests pyyaml pandas-market-calendars tzlocal tqdm
    pip install lightgbm xgboost
    # macOS only
    brew install libomp
  5. Verify incremental updates with test_incremental_update.py

    master

    To ensure that the data fetching and storage logic correctly handles holidays, weekends, and data gaps, run the provided test script:

    python test_incremental_update.py

    This test verifies:

    1. Correctness of data fetching.
    2. Handling of weekends.
    3. Handling of public holidays.
    4. Detection of data gaps.
    5. Correctness of data storage.
    6. Integrity of the incremental update process.
  6. Deploy FinRL-X using deploy.sh

    master

    Use the deploy.sh script for an automated workflow that handles dependency checks, data downloading, and strategy execution. This is the fastest way to run backtests or paper trading.

    Common commands:

    • Backtest: ./deploy.sh --strategy <strategy_name> --mode backtest
    • Custom Date Range: ./deploy.sh --strategy <strategy_name> --mode backtest --start <YYYY-MM-DD> --end <YYYY-MM-DD>
    • Single Date Signal: ./deploy.sh --strategy <strategy_name> --mode single --date <YYYY-MM-DD>
    • Paper Trading (Preview): ./deploy.sh --strategy <strategy_name> --mode paper --dry-run
    • Paper Trading (Execute): ./deploy.sh --strategy <strategy_name> --mode paper
    # Backtest (downloads data + runs strategy)
    ./deploy.sh --strategy adaptive_rotation --mode backtest
    
    # Custom date range
    ./deploy.sh --strategy adaptive_rotation --mode backtest --start 2020-01-01 --end 2025-12-31
    
    # Single date signal
    ./deploy.sh --strategy adaptive_rotation --mode single --date 2024-12-31
    
    # Paper trading (requires Alpaca credentials in .env)
    ./deploy.sh --strategy adaptive_rotation --mode paper --dry-run   # preview
    ./deploy.sh --strategy adaptive_rotation --mode paper              # execute
  7. Recompute y_return in fundamental_data

    master

    If trade_price has been updated in the database, you must recompute all y_return values to maintain data integrity. This script iterates through the fundamental_data table and updates the y_return column based on the new trade_price and the subsequent trade_price for each ticker.

    import sqlite3, pandas as pd, numpy as np
    conn = sqlite3.connect('data/finrl_trading.db')
    df = pd.read_sql('SELECT ticker, datadate, trade_price FROM fundamental_data ORDER BY ticker, datadate', conn)
    df['trade_price'] = pd.to_numeric(df['trade_price'], errors='coerce')
    df['next_tp'] = df.groupby('ticker')['trade_price'].shift(-1)
    df['y_return_new'] = np.where(
        (df['trade_price'] > 0) & (df['next_tp'] > 0),
        np.log(df['next_tp'] / df['trade_price']), np.nan
    )
    cursor = conn.cursor()
    for _, row in df.iterrows():
        val = None if pd.isna(row['y_return_new']) else round(float(row['y_return_new']), 6)
        cursor.execute('UPDATE fundamental_data SET y_return = ? WHERE ticker = ? AND datadate = ?',
                       (val, row['ticker'], row['datadate']))
    conn.commit()
    conn.close()
  8. Configure API keys in .env

    master

    Create a .env file in the project root to configure trading credentials and data sources.

    Required for trading (Alpaca):

    • APCA_API_KEY
    • APCA_API_SECRET
    • APCA_BASE_URL (Use https://paper-api.alpaca.markets for paper trading)

    Optional for higher data quality:

    • FMP_API_KEY (Financial Modeling Prep)
    • WRDS_USERNAME and WRDS_PASSWORD (Wharton Research Data Services)
    # Required for trading
    APCA_API_KEY=your_alpaca_key
    APCA_API_SECRET=your_alpaca_secret
    APCA_BASE_URL=https://paper-api.alpaca.markets
    
    # Optional for better data quality
    FMP_API_KEY=your_fmp_key
    WRDS_USERNAME=your_wrds_username
    WRDS_PASSWORD=your_wrds_password
  9. Install trading calendar dependencies

    master

    To use the trading calendar functionality (e.g., for NYSE/NASDAQ holiday handling), you must install the underlying calendar libraries. You can install them individually or via the project's requirements file.

    Individual Installation

    • For pandas-market-calendars:
    pip install pandas-market-calendars
    • For exchange-calendars:
    pip install exchange-calendars

    Bulk Installation

    pip install -r requirements.txt
  10. Explore FinRL-X Strategy Use Cases

    master

    FinRL-X provides three primary strategy paradigms:

    1. Portfolio Allocation Paradigms: Compares different allocation methods (Equal Weight, Mean-Variance, Minimum Variance, DRL Allocator, and KAMA Timing) using a unified weight-vector interface.
    2. Rolling Stock Selection + DRL: Combines quarterly NASDAQ-100 stock selection (via ML fundamental scoring) with DRL-based portfolio allocation, utilizing strict no-lookahead semantics.
    3. Adaptive Multi-Asset Rotation: A walk-forward-safe strategy that rotates between Growth Tech, Real Assets, and Defensive asset groups based on Information Ratio and market regimes (Slow/Fast risk-off). It includes risk controls like trailing stop-losses and cooldown periods.
  11. Use Mixed-Vintage mode for live inference

    master
    When using --mixed-vintage, the pipeline does not assume a single uniform datadate for all stocks. Instead, it uses the latest available data per current-SP500 ticker. This accounts for companies reporting earnings at different times (e.g., some may have Q1 2026 data while others only have Q4 2025 data). All inference tickers are then ranked together within their respective buckets.