easy-tdx

repository·main·Indexed 20 days ago

https://github.com/handsomejustin/easy_tdx

A high-performance financial data tool and TCP protocol client for TongdaXing, providing access to market data for A-shares, HK, US, and Futures. It supports real-time quotes, K-lines, tick data, and F10 company information. The library includes a Python API, a CLI, and a Web API, featuring a Screen Strength tool to rank stocks based on weighted returns and volatility across different time cycles.

Tokens
125.8K
Snippets
400
Records
528
Agent score
68%

What's inside easy-tdx

  1. Overview of Web UI features (MVP-A)

    main

    The current version of the Web UI provides the following capabilities for single-asset strategy backtesting:

    • Market Data Selection: Fetch data for Shanghai, Shenzhen, and Beijing markets across Daily, Weekly, and Minute intervals via the /api/v1/bars endpoint.
    • Strategy Configuration: Select from preset strategies (e.g., MA Crossover, MACD, Bollinger Bands, RSI, KDJ). The parameter forms are dynamically rendered based on the backend schema.
    • Backtest Reporting:
      • K-Line Chart: Main chart with buy/sell point annotations using ECharts candlestick and markPoint.
      • Equity Curve: Dual-axis chart showing net value and drawdown.
      • Performance Metrics: A table containing 19 performance indicators (covering returns, risk, and trade groupings).
      • Trade Records: A detailed table of all executed trades.
  2. Overview of the v1.11.0 Factor Engine

    main

    The v1.11.0 Factor Engine introduces a systematic way to calculate and manage quantitative factors within easy-tdx. It utilizes an ABC (Abstract Base Class) and Registry pattern, ensuring consistency with existing indicator patterns in the repository.

    Key capabilities include:

    • Factor Base & Registry: A structured way to define and register new factors.
    • FactorEngine: A calculation engine supporting two modes: single-stock multi-factor calculation and cross-stock sectional calculation.
    • Built-in Factors: 15 pre-implemented factors that bridge existing capabilities from MyTT and ChanlunAnalyser.
    • CLI Integration: The easy-tdx factor list command to enumerate available factors.

    All calculations are performed using pure numpy vectorization and do not depend on network calls, ensuring high performance.

  3. Overview of the easy-tdx Backtest Engine

    main

    The easy_tdx.backtest module is a pure-computation backtesting engine designed for vectorized strategy execution. It is built to be independent of network dependencies, allowing for high-performance local testing of trading strategies.

    Key Features:

    • Strategy Definition: Supports Python class-based strategy definitions.
    • Execution: Vectorized execution pipeline.
    • Matching Rules: Supports 5 different matching/order execution rules.
    • Performance Metrics: Provides 18 distinct performance indicators.
    • CLI Integration: One-click backtesting via command line.

    Architecture Layers:

    1. Data Types (types.py)
    2. Strategy Base Class & Data Proxy (strategy.py)
    3. Order Matcher (orders.py)
    4. Portfolio Tracking (portfolio.py)
    5. Performance Analysis (performance.py)
    6. Engine Orchestration (engine.py)
    7. CLI Integration (cli.py)
  4. Implement factor analysis and preprocessing in easy-tdx

    main

    The v1.12.0 plan introduces quantitative factor engine capabilities to easy-tdx. This involves two main components:

    1. Preprocessing (factor/transform.py): A set of 6 pure functions that take a pandas.DataFrame as input and return a processed pandas.DataFrame. These functions handle data cleaning and normalization.
    2. Analysis (factor/analysis.py): The FactorAnalyzer class, which processes long-format cross-sectional data to calculate metrics such as Information Coefficient (IC), factor grouping (deciles/quintiles), and decay.

    Data Requirements:

    • factor_data: A DataFrame containing columns [date, code, factor_name1, factor_name2, ...].
    • return_data: A DataFrame containing columns [date, code, forward_Nd] (where forward_Nd represents the N-day forward return).

    CLI Access: New functionality is accessible via the command: easy-tdx factor analyze.

  5. Overview of easy-tdx Architecture

    main

    The easy-tdx project is structured into several functional layers:

    • Clients: TdxClient (standard), MacClient (MAC protocol), and ExTdxClient (extended markets).
    • Core Logic: indicator.py (technical indicators based on MyTT), chanlun (technical analysis), and factor (factor engine).
    • Strategy & Backtesting: backtest (engine), portfolio (management), and screen (scanning).
    • Real-time & Web: realtime (event-driven data push) and web (FastAPI REST/WebSocket API).
    • Infrastructure: transport (sync/async connections), codec (data encoding/decoding), and models (dataclasses).
  6. How SlippageModel and ExecutionModel work together

    main

    The backtesting engine uses a decoupled architecture for simulating market reality:

    1. OrderSimulator manages the high-level backtest loop and state (cash, position). It uses a SlippageModel to calculate the cost of a trade based on current market conditions (price, volume, and volatility).
    2. ExecutionModel (e.g., ImmediateExecution) defines the logic of when and how a signal becomes a trade. It takes a SlippageModel as an input to ensure that the simulated execution price includes the calculated slippage cost.

    This separation allows you to swap out different execution strategies (e.g., immediate vs. scheduled) while keeping the same market impact (slippage) logic.

  7. Understand TDX Price Differential Encoding

    main

    TDX does not return absolute prices in many responses (like real-time quotes). Instead, it uses differential encoding (delta encoding). Most price fields are calculated as an offset from a base value price_raw.

    Real-time Quotes (security_quotes.py): All fields are relative to price_raw and must be divided by 100.0 to get the actual price:

    • pre_close = (price_raw + last_close_diff) / 100.0
    • open = (price_raw + open_diff) / 100.0
    • high = (price_raw + high_diff) / 100.0
    • bid1 = (price_raw + bid1_d) / 100.0

    K-Line Data (security_bars.py): K-line data uses cumulative differential encoding. Each bar's absolute values are derived from the previous bar's closing value:

    • open_abs = open_diff + pre_diff_base
    • close_abs = open_abs + close_diff
    • high_abs = open_abs + high_diff
    • low_abs = open_abs + low_diff
    • pre_diff_base = open_abs + close_diff (This becomes the base for the next K-line bar).
  8. How the Factor system and registration works

    main

    The factor system is built on a base Factor class and a central FACTORY_REGISTRY.

    1. Registration: Factors are defined as classes and decorated with @register_factor. This automatically adds them to the FACTORY_REGISTRY during module import.
    2. Discovery: The easy_tdx.factor.builtin module imports all sub-modules (momentum, volatility, etc.) to trigger this registration process. This makes all built-in factors available via list_factors() and get_factor().
    3. Computation: Every factor class must implement a .compute(df) method that takes a DataFrame and returns a pd.Series of the same length.
  9. Understand the TDX Protocol Frame Structure

    main

    The TDX protocol uses raw TCP (typically on port 7709) without HTTP wrapping. Communication follows a specific frame format discovered via packet sniffing:

    Response Frame Structure: 16 bytes fixed header + variable length body

    Header Layout (from codec/frame.py):

    • Offset 0: 4 bytes (Unknown)
    • Offset 4: 4 bytes (Unknown)
    • Offset 8: 4 bytes (Unknown)
    • Offset 12: 2 bytes (zipsize) — The actual length of the body.
    • Offset 14: 2 bytes (unzipsize) — The length of the body after decompression.

    Decompression Logic: If zipsize == unzipsize, the body is uncompressed. Otherwise, the body must be decompressed using zlib.

    # Conceptual representation of the frame header
    # Offset 12: H (2 bytes) -> zipsize
    # Offset 14: H (2 bytes) -> unzipsize
  10. How the Factor system and registry work together

    main

    The factor system is built on a plugin-style architecture using an Abstract Base Class (ABC) and a global registry:

    1. Contract: The Factor class defines the mandatory interface (name, category, inputs, and compute).
    2. Validation: The __init_subclass__ hook in Factor ensures that any new subclass defines all required attributes at the moment of class definition, preventing runtime errors during computation.
    3. Discovery: The @register_factor decorator populates FACTORY_REGISTRY. This registry acts as a lookup table that maps string names to class types.
    4. Execution: The FactorEngine uses the registry to resolve string names into executable logic, allowing users to request factors by name (e.g., via CLI or API) without needing to pass class objects manually.
  11. Backtest Engine Specification Overview

    main

    The easy_tdx backtest engine is designed around several core components and requirements:

    Core Data Types

    • Signal/Trade/Position/BacktestResult: Structured data types for tracking strategy execution.
    • Signal.direction: Uses Literal["BUY", "SELL"].
    • Trade.rejected: A bool flag.
    • BacktestResult.performance: A dict[str, float] containing performance metrics.

    Strategy Implementation

    • Strategy Base Class: Requires init() and next() methods.
    • StrategyDataProxy: Provides access to data and supports pre-computed columns.
    • crossover(): A helper function for signal detection.
    • @dsl_strategy: A decorator for defining strategies via DSL.

    Execution and Management

    • Execution Modes: Supports 5 different execution modes.
    • Position Management: Supports full, fixed, and percent modes.
    • Order Rejection: Supports reduce and skip strategies.
    • Fee Models: Includes support for commissions, stamp duty, and slippage.
    • PortfolioTracker: Tracks equity curves and calculates drawdowns.

    Engine and CLI

    • BacktestEngine: Operates via a four-step pipeline and calculates PnL.
    • CLI: Supports automatic K-line fetching, --indicators for pre-computation, and multiple output formats (JSON, table, csv).
  12. Access K-line data and precomputed indicators via StrategyDataProxy

    main

    Inside a Strategy class, the self.data property provides a StrategyDataProxy. This proxy allows you to access standard K-line columns (open, close, high, low, vol, amount) and any additional precomputed indicator columns present in the input DataFrame.

    Use index notation to access specific bars:

    • self.data.close[0]: The current bar's close price.
    • self.data.close[-1]: The previous bar's close price.

    If your DataFrame contains columns like MACD_DIF or BOLL_UPPER (from get_stock_kline_with_indicators), they are automatically exposed via __getattr__.

    class BollingerBreakout(Strategy):
        def init(self):
            pass  # Indicators are already in the DataFrame
    
        def next(self):
            # Accessing precomputed columns directly
            if self.data.close[0] > self.data.BOLL_UPPER[0]:
                self.sell(size=0)
            elif self.data.close[0] < self.data.BOLL_LOWER[0]:
                self.buy(size=0)