DOCSEARCHbyAlgolia
Submit Docs

polars_ta

repository·main·Indexed 42 days ago

https://github.com/wukan1986/polars_ta

A library of technical indicator operators for the Polars DataFrame library. It provides Polars Expr-native wrappers for common financial indicators, including implementations that mimic WorldQuant Alpha (wq), TA-Lib, and TongDaXin (tdx) naming conventions. The library allows technical analysis functions to be used directly within Polars expression contexts such as .with_columns() or .select().

Tokens
5K
Snippets
12
Records
28
Agent score
54%

What's inside polars_ta

  1. Overview of polars_ta.talib

    main
    The polars_ta.talib package is a wrapper around talib functions designed specifically for use with the Polars library. Unlike the standard talib which operates on Series, this package uses Polars Expr for both inputs and outputs, allowing these technical analysis functions to be used directly within Polars expression contexts (e.g., inside .with_columns() or .select()).

  2. Understand Point In Time (PIT) data structures

    main

    Point In Time (PIT) data management is used to handle corrections to historical erroneous data in both market and financial statement datasets. The data structures differ based on the data type:

    • Market Data: Structured as date\asset\features\[update_time]. In market data, date is often omitted because the update time typically coincides with the report period. If duplicates exist, the update_time field is used to distinguish them.
    • Financial Statement Data: Structured as date\asset\features\update_time. Here, date represents the report period, and update_time represents the actual release/announcement date.

  3. Understand the design philosophy of polars_ta

    main

    The polars_ta library provides functions designed to closely mimic the WorldQuant Alpha101 style, but with specific adjustments to improve usability and compatibility with expression trees (e.g., for genetic programming).

    There are two ways to approach function usage in this library:

    1. Chained Method Style: Using methods directly on a Polars expression (e.g., x.abs().log()). This is convenient for IDE auto-completion but can lead to inconsistent input patterns.
    2. Functional Wrapper Style: Using functions that wrap expressions (e.g., log(abs_(x))). This style is preferred for genetic programming and expression tree construction because it uses a consistent parameter list and allows for easy nesting.

    Note on Naming: To avoid conflicts with Python built-in functions, certain names are suffixed with an underscore. For example:

    • Use abs_ instead of abs
    • Use and_ instead of and
    • Use int_ instead of int
    • Use max_ instead of max

  4. Understand the `polars_ta.tdx` module conventions

    main

    The polars_ta.tdx module implements technical analysis functions following the tdx (TongDaXin) naming convention.

    When using this module, be aware of the following:

    • Function Types: Most functions are time-series functions. However, some are element-wise functions. Pay close attention to functions like MAX to ensure you are using the intended version for your calculation type.
    • Import Priority: The library follows a specific hierarchy for function implementation to ensure consistency. When looking for a function, the implementation priority is:
      1. tdx (Primary implementation)
      2. wq (Secondary implementation)
      3. ta (Fallback implementation)

    Developers should prioritize importing from tdx to maintain compatibility with the tdx naming standard.

  5. Understand the types of technical indicators in polars_ta

    main

    The library categorizes technical indicators into three distinct types based on their calculation logic and data requirements:

    1. Time series indicators (ts_): These require data to be sorted by time before calculation. They are more complex to handle when null values appear in the middle of the series.
    2. Cross-sectional indicators (cs_): These operate on a per-row basis across multiple columns. They do not require a specific order and must be compatible with null values.
    3. Single element indicators: These can be calculated using single or multiple columns in any manner.

    Relationship between ts_ and cs_ indicators: ts_ indicators are essentially cs_ indicators applied over a rolling time window. For example, ts_rank is implemented by calculating cs_rank within a rolling window and taking the last value ([-1]) for each step.

    Performance Note: To maintain high performance, polars_ta avoids calling cs_ functions repeatedly within a Python-level rolling loop (which would cause massive overhead). Instead, rolling operations are typically implemented in low-level languages like Cython or Numba to call the underlying cs_ logic efficiently.

  6. How polars_ta indicator modules are organized

    main

    The library organizes indicators into different modules based on their implementation style and priority. When naming conflicts occur, you should call them in the following order of priority (higher priority means the implementation is closer to a native Polars Expr):

    1. wq: Mimics WorldQuant Alpha formulas. This is the highest priority.
    2. ta: A polars-style version of TA-Lib. It attempts to reuse functions from wq.
    3. tdx: Implements indicators often found in tdx systems, also attempting to import from wq and ta.
    4. talib: Maintains the exact same signatures and parameters as the original TA-Lib.

  7. Understand EMA compatibility modes in polars_ta

    main

    The polars_ta library only implements EMA using compatibility mode 1. This mode is consistent with how Chinese stock software calculates EMA.

    In TA-Lib, there are two modes:

    • Compatibility Mode 0 (Default in TA-Lib): The first non-NaN value is initialized as talib.SMA(CLOSE, N). This mode is complex and inefficient to implement via Polars expressions, so it is not supported in polars_ta.
    • Compatibility Mode 1 (Supported by polars_ta): The first non-NaN value is simply the CLOSE price.

    Because EMA is a building block for many indicators, using polars_ta will affect the calculation of MACD, DEMA, TEMA, TRIX, and T3.

  8. Evolution of TA-Lib wrappers in polars_ta

    main

    The project has evolved through four stages of implementation to balance ease of use, performance, and compatibility with automated factor mining (like genetic algorithms):

    1. Expr.map_batches Wrapper: The initial approach using Polars' native mapping. Requires wrapping third-party APIs (like TA-Lib) into functions. Inputs/outputs are limited to single columns; multi-column support requires converting to pl.Struct and using unnest later.
    2. register_expr_namespace: Simplifies usage by allowing member function calls.
      • Pros: Easy to use.
      • Cons: Difficult to pass into genetic algorithms; loses IDE support due to dynamic __getattribute__ calls.
    3. Prefix Expressions: Converts member functions into formulas.
      • Pros: Compatible with genetic algorithms.
      • Cons: Still loses IDE support due to dynamic method calls.
    4. Code Generation (Current Approach): Uses codegen_talib.py to generate the polars_ta/talib/__init__.py file.
      • Pros: Compatible with genetic algorithms and provides full IDE support/autocompletion.

  9. Understand the difference between Null and NaN in Polars

    main

    It is critical to distinguish between null and NaN when working with Polars (which uses the Arrow backend):

    • Null: Represents a missing value (equivalent to None in Python). Polars uses a validity bitmap array to track nulls, making null-counting operations very efficient.
    • NaN: Represents a floating-point 'Not a Number' (e.g., from 0/0 or log(-1)).

    Key differences:

    • None == None is True.
    • np.nan == np.nan is False.
    • Most Polars functions adapt to null but not NaN. Always use .fill_nan(None) to convert NaN to null for better compatibility.

  10. Understand SMA (Chinese Version) and RMA implementation

    main

    The Chinese version of SMA(X, N, M) is conceptually similar to RMA compatibility mode 0 (where the first valid value is a moving average).

    However, polars_ta implements this using RMA compatibility mode 1 to maintain efficiency. Because of this difference in initialization, there may be data discrepancies in the early part of a series.

    Important: To minimize the impact of these discrepancies and ensure accurate results (especially for unit testing), always provide a sufficient length of initial data.

    This implementation affects indicators like ATR and RSI.

  12. Build polars_ta from source

    main

    To build from source, clone the repository, build the package using python -m build, and install the resulting wheel file.

    git clone --depth=1 https://github.com/wukan1986/polars_ta.git
cd polars_ta
python -m build
cd dist
pip install polars_ta-0.1.2-py3-none-any.whl