Qlib Quantitative Investment Platform

Qlib is an open-source, AI-oriented quantitative investment platform designed to empower research and create value using AI technologies. It covers the entire quantitative investment chain, including alpha seeking, risk modeling, portfolio optimization, and order execution.

Key capabilities include:

• Full ML Pipeline: Supports data processing, model training, and back-testing.
• Diverse Modeling Paradigms: Supports supervised learning (for mining non-linear patterns), market dynamics modeling (using adaptive concept drift), and reinforcement learning (for continuous investment decisions).
• SOTA Research: Integrates state-of-the-art quantitative research works and models.

Developers contributing to Qlib must adhere to the following standards:

• Docstrings: Use the Numpydoc Style for all docstrings.
• Formatting: Code must align with black formatting standards.
• Linting: Code must pass pylint and flake8 checks as defined in the CI workflow.

Qlib provides several built-in scripts for collecting different types of financial data:

• yahoo: Retrieves US and CN stock data from Yahoo Finance.
• fund: Retrieves fund data from http://fund.eastmoney.com.
• cn_index: Retrieves Chinese indices (e.g., CSI300, CSI100) from http://www.csindex.com.cn.
• us_index: Retrieves US indices (e.g., SP500, NASDAQ100, DJIA, SP400) from Wikipedia.
• contrib: Contains scripts for auxiliary functions.

Qlib includes a custom implementation of Graph Attention Networks (GATs) built using PyTorch. GATs use masked self-attentional layers on graph-structured data, allowing nodes in stacked layers to attend to their neighborhoods' features. This implementation avoids costly matrix operations (like inversion) and does not require the graph structure to be known upfront.

For theoretical background, refer to the original paper: Graph Attention Networks.

A Point-in-Time (PIT) database is designed to prevent data leakage during historical market analysis and backtesting. In financial datasets, information (like financial reports) is often amended or updated over time. If a backtest uses the latest version of a data point for a historical timestamp, it is using information that would not have been available at that time.

A PIT database ensures that for any historical timestamp, the user retrieves the specific version of the data that was actually available at that moment, maintaining consistency between historical backtesting and online trading performance.

Temporally Correlated Task Scheduling (TCTS) is a framework for sequence learning that utilizes multiple temporally correlated auxiliary tasks to improve the training of a main task. In financial contexts like stock trend forecasting, these auxiliary tasks might involve predicting stock prices for different future time steps (e.g., predicting tomorrow's price vs. the day after tomorrow).

Key components include:

• Learnable Task Scheduler: An adaptive component that selects the best auxiliary task during training based on the current model status and minibatch data.
• Bi-level Optimization: The scheduler and the main model are trained jointly. The scheduler is optimized to maximize validation performance, while the model is optimized to minimize training loss guided by the scheduler's task selection.

DoubleEnsemble is an ensemble framework designed for financial market prediction. It addresses two primary challenges in financial data: low signal-to-noise ratios and the increasing number of features.

It works through two main mechanisms:

1. Sample Reweighting: Identifies key samples based on training dynamics (learning trajectories).
2. Feature Selection: Identifies key features using shuffling-based ablation impact.

The framework is model-agnostic and can be applied to a wide range of base models to extract complex patterns while mitigating overfitting and instability.

State-Frequency-Memory (SFM) is a recurrent network architecture designed for stock price prediction. It utilizes the Discrete Fourier Transform (DFT) to decompose the hidden states of memory cells. This allows the model to capture multi-frequency trading patterns from historical market data, which are then used to predict future stock prices.

For more details on the underlying theory, refer to the paper: Stock Price Prediction via Discovering Multi-Frequency Trading Patterns.