jqfactor_analyzer

repository·master·Indexed 20 days ago

https://github.com/joinquant/jqfactor_analyzer

An open-source tool designed to work with jqdatasdk for attribution analysis, factor data caching, and single-factor analysis. It implements multi-factor risk model theory to decompose stock returns into style, industry, country, and specific returns. Key features include the AttributionAnalysis class for portfolio exposure and benchmark relative analysis, the analyze_factor() function for comprehensive single factor evaluation, and a local caching system to optimize data retrieval from JoinQuant.

Tokens
6.8K
Snippets
22
Records
27
Agent score
22%

What's inside jqfactor_analyzer

  1. Concepts of Multi-Factor Risk Models

    master

    The attribution analysis is based on the multi-factor risk model theory, which posits that stock returns are driven by common factors. The model decomposes returns into:

    1. Style Factors: Factors affecting returns such as size, growth, leverage, etc.
    2. Industry Factors: Returns associated with specific industry sectors.
    3. Country Factors: The overall market movement affecting all stocks in the same market.
    4. Specific Returns (Idiosyncratic Returns): The portion of returns that cannot be explained by the common factors (e.g., company-specific news, management decisions).

    The mathematical representation of stock return $R_i$ is: $R_i = \text{Country Factor Return} + \sum \text{Style Factor Returns} + \sum \text{Industry Factor Returns} + \text{Specific Return}$

  2. Configure the factor cache directory

    master

    The factor_cache module provides local data caching to speed up analysis and reduce server requests. By default, data is cached in the user's home directory under ~/jqfactor_datacache/bundle. You can change this path using set_cache_dir. Note that the cache uses pyarrow.feather format; if you encounter corruption due to pyarrow version changes, delete the cache directory and re-cache.

    from jqfactor_analyzer.factor_cache import set_cache_dir, get_cache_dir
    set_cache_dir(my_path) # Set the cache directory to my_path
    print(get_cache_dir()) # Output the current cache directory
  3. Format custom factor data for analysis

    master

    To use your own factor data with jqfactor_analyzer, you must convert it into a pandas.DataFrame that meets these requirements:

    1. Index: Must be a pandas.DatetimeIndex sorted in ascending order.
    2. Columns: Must be stock codes following JoinQuant rules:
      • Shenzhen stocks: Append .XSHE (e.g., 000001.XSHE).
      • Shanghai stocks: Append .XSHG (e.g., 600000.XSHG).

    Example Conversion:

    import pandas as pd
    
    sample_data = pd.DataFrame(
        [[0.84, 0.43], [1.06, 0.51]],
        index=['2018-01-02', '2018-01-03'],
        columns=['000001.XSHE', '000002.XSHE']
    )
    
    # 1. Convert index to DatetimeIndex
    sample_data.index = pd.to_datetime(sample_data.index)
    
    # 2. Sort index
    sample_data = sample_data.sort_index()
    
    # 3. Verify stock code format
    if not sample_data.columns.astype(str).str.match('\d{6}\.XSH[EG]').all():
        print("Warning: Invalid stock code format")
    
    factor_data = sample_data
    import pandas as pd
    
    sample_data = pd.DataFrame(
        [[0.84, 0.43, 2.33, 0.86, 0.96],
         [1.06, 0.51, 2.60, 0.90, 1.09]],
        index=['2018-01-02', '2018-01-03'],
        columns=['000001.XSHE', '000002.XSHE', '000063.XSHE', '000069.XSHE', '000100.XSHE']
    )
    
    factor_data = sample_data.copy()
    factor_data.index = pd.to_datetime(factor_data.index)
    factor_data = factor_data.sort_index()
  4. Fetch factor data using jqdatasdk

    master

    To use real market data from the JoinQuant factor library, use the jqdatasdk library. You must authenticate with your JoinQuant credentials.

    1. Install/Import jqdatasdk.
    2. Authenticate using jqdatasdk.auth('username', 'password').
    3. Use jqdatasdk.get_factor_values() to retrieve specific factors for a set of securities over a date range.
    import jqdatasdk
    
    # Authenticate with JoinQuant credentials
    jqdatasdk.auth('username', 'password')
    
    # Example: Fetching VOL5 (5-day average turnover) for CSI 300 stocks
    factor_data = jqdatasdk.get_factor_values(
        securities=jqdatasdk.get_index_stocks('000300.XSHG'),
        factors=['VOL5'],
        start_date='2018-01-01',
        end_date='2018-12-31'
    )['VOL5']
  5. Cache and retrieve factor data groups

    master

    To avoid repeated expensive API calls to jqdatasdk, you can save factor values to a local cache as a group.

    1. Save/Check Cache: Use save_factor_values_by_group to download and store a group of factors for a specific date range. If overwrite=False, it will check for existing data first.
    2. Retrieve from Cache: Use get_factor_values_by_cache to load the data back into a pandas DataFrame.

    Note on parameters: When retrieving data, if you provided a group_name during the saving process, the factor_names parameter becomes redundant and will be ignored by the cache lookup.

    from jqfactor_analyzer.factor_cache import save_factor_values_by_group, get_factor_values_by_cache, get_cache_dir
    import jqdatasdk as jq
    import pandas as pd
    
    # 1. Setup parameters
    # Assuming jq.auth() has been called to authenticate jqdatasdk
    all_factors = jqdatasdk.get_all_factors()
    factor_names = all_factors[all_factors.category=='growth'].factor.tolist()
    group_name = 'growth_factors'
    start_date = '2021-01-01'
    end_date = '2021-06-01'
    
    # 2. Save/Check cache
    # Returns the path to the cached folder
    factor_path = save_factor_values_by_group(
        start_date, 
        end_date, 
        factor_names=factor_names, 
        group_name=group_name, 
        overwrite=False, 
        show_progress=True
    )
    
    # 3. Retrieve and concatenate cached data
    trade_days = jqdatasdk.get_trade_days(start_date, end_date)
    factor_values_list = []
    for date in trade_days:
        # Retrieve data for each date from the cache
        df = get_factor_values_by_cache(
            date, 
            codes=None, 
            factor_names=factor_names, 
            group_name=group_name, 
            factor_path=factor_path
        )
        factor_values_list.append(df)
    
    # Combine into a single DataFrame
    factor_values = pd.concat(factor_values_list)
    print(factor_values.head())
  6. Perform Attribution Analysis with AttributionAnalysis

    master

    To perform attribution analysis, use the jqfactor_analyzer.AttributionAnalysis class. You need to provide portfolio weight information and daily returns.

    Data Requirements:

    • Weight Data: A pandas DataFrame where the index is the date, columns are stock codes (use jqdatasdk.normalize_code for formatting), and values are weights. Daily weights should sum to $\le 1$.
    • Daily Returns: A pandas Series where the index is the date and values are daily returns.

    Initialization Example:

    import jqdatasdk
    import jqfactor_analyzer as ja
    
    # Authenticate with jqdatasdk
    jqdatasdk.auth("YOUR_USERNAME", "YOUR_PASSWORD")
    
    # Initialize AttributionAnalysis
    # style_type: 'style' or 'industry'
    # industry: e.g., 'sw_l1'
    # use_cn: boolean
    An = ja.AttributionAnalysis(weight_infos, daily_return, style_type='style', industry='sw_l1', use_cn=True, show_data_progress=True)
    import jqdatasdk
    import jqfactor_analyzer as ja
    
    # Authenticate with jqdatasdk
    jqdatasdk.auth("YOUR_USERNAME", "YOUR_PASSWORD")
    
    # Initialize AttributionAnalysis
    An = ja.AttributionAnalysis(weight_infos, daily_return, style_type='style', industry='sw_l1', use_cn=True, show_data_progress=True)
  7. Perform Attribution Analysis Relative to a Benchmark Index

    master

    To analyze how your portfolio performs relative to a specific index (benchmark), use the following methods:

    1. Relative Exposure: Use get_exposure2bench(index_symbol) to find the difference between portfolio exposure and index exposure.
    2. Relative Daily Returns: Use get_attr_daily_returns2bench(index_symbol) to get the daily return decomposition relative to the benchmark. This method aligns portfolio and index positions and accounts for cash returns (where cash return relative to index is calculated as $(-1) \times \text{remaining position} \times \text{index return}$).
    3. Relative Cumulative Returns: Use get_attr_returns2bench(index_symbol) to get the cumulative returns relative to the benchmark, accounting for compounding effects.
    # Get exposure relative to benchmark
    rel_exposure = An.get_exposure2bench('000905.XSHG')
    
    # Get daily returns relative to benchmark
    rel_daily_returns = An.get_attr_daily_returns2bench('000905.XSHG')
    
    # Get cumulative returns relative to benchmark
    rel_cum_returns = An.get_attr_returns2bench('000905.XSHG')
  8. Fetch JoinQuant factor data via jqdatasdk

    master

    You can retrieve pre-built factors from the JoinQuant library using jqdatasdk.

    Steps:

    1. Authenticate using your JoinQuant credentials.
    2. Use get_factor_values to fetch the data.
    import jqdatasdk
    
    # Authenticate
    jqdatasdk.auth('username', 'password')
    
    # Fetch VOL5 (5-day average turnover) for CSI 300 stocks
    factor_data = jqdatasdk.get_factor_values(
        securities=jqdatasdk.get_index_stocks('000300.XSHG'),
        factors=['VOL5'],
        start_date='2018-01-01',
        end_date='2018-12-31'
    )['VOL5']
    import jqdatasdk
    jqdatasdk.auth('username', 'password')
    
    # 获取聚宽因子库中的VOL5数据
    factor_data=jqdatasdk.get_factor_values(
        securities=jqdatasdk.get_index_stocks('000300.XSHG'),
        factors=['VOL5'],
        start_date='2018-01-01',
        end_date='2018-12-31')['VOL5']
  9. Perform single factor analysis with analyze_factor()

    master

    Use analyze_factor() to perform a comprehensive analysis of a single factor. The function accepts factor data in pandas.DataFrame format (index as DatetimeIndex, columns as stock codes following JoinQuant rules like 000001.XSHE) or pd.Series format (MultiIndex of date and stock code).

    Key Parameters:

    • factor: The factor values (DataFrame or Series).
    • industry: Industry classification. Options: 'sw_l1', 'sw_l2', 'sw_l3' (Shenwan), 'jq_l1', 'jq_l2' (JoinQuant), or 'zjw' (CSRC).
    • quantiles: Number of groups for factor value partitioning (default: 5).
    • periods: Rebalancing periods (e.g., [1, 5, 10]).
    • weight_method: Weighting method for quantile returns. Options: 'avg' (equal weight), 'mktcap' (total market cap), 'ln_mktcap' (log total market cap), 'cmktcap' (circulating market cap), 'ln_cmktcap' (log circulating market cap).
    • max_loss: Maximum allowed percentage of invalid factor data (NaNs, duplicates, etc.) to be discarded (default: 0.25).
    • allow_cache: Whether to allow local caching of price and market cap data.
    • show_data_progress: Whether to show data retrieval progress.
    from jqfactor_analyzer import analyze_factor
    
    # Perform analysis
    far = analyze_factor(
        factor_data, 
        quantiles=10, 
        periods=(1, 10), 
        industry='jq_l1', 
        weight_method='avg', 
        max_loss=0.1
    )
  10. Configure the factor data cache directory

    master

    You can manage where factor data is stored locally using set_cache_dir. This is useful for directing large datasets to specific drives or folders. Use get_cache_dir to verify the current active cache path.

    from jqfactor_analyzer.factor_cache import set_cache_dir, get_cache_dir
    
    # Set the cache directory to a specific path
    my_path = 'E:\\jqfactor_cache'
    set_cache_dir(my_path)
    
    # Verify the current cache directory
    print(get_cache_dir())