QuantsLab

repository·main·Indexed 43 days ago

https://github.com/hummingbot/quants-lab

A Python quantitative research framework for Hummingbot trading strategies. Version 2.0.0 provides tools for data collection, backtesting, strategy development, and automated task deployment. It includes a CLI for managing task lifecycles via YAML configurations, MongoDB integration for data storage, and a Smart Bot Orchestration system for automating grid trading based on market signals and performance metrics.

Tokens
33.3K
Snippets
95
Records
122
Agent score
55%

What's inside QuantsLab

  1. Deploy a Recurring Task

    main

    To run automated tasks in production, it is recommended to use Docker. Follow these steps to activate the environment, start the database, and run your task configuration file.

    Production Deployment (Docker):

    1. Activate the Conda environment: conda activate quants-lab
    2. Start the MongoDB database: make run-db
    3. Run the tasks: make run-tasks config=FILE.yml
    4. Monitor logs and processes: make logs-tasks and make ps-tasks
    5. Stop tasks when finished: make stop-tasks

    Local Development Mode: To run tasks locally without Docker, use the source=1 flag:

    make run-tasks config=tf_pipeline.yml source=1
    # Production deployment sequence
    conda activate quants-lab
    make run-db
    make run-tasks config=tf_pipeline.yml
    make logs-tasks
    make ps-tasks
    make stop-tasks
  2. Install QuantsLab

    main

    To set up the QuantsLab environment, clone the repository and use the provided Makefile. The installer automatically configures a Conda environment (Python 3.12), installs all dependencies, sets up a MongoDB database, and generates necessary configuration files.

    git clone https://github.com/hummingbot/quants-lab.git
    cd quants-lab
    make install
  3. Configure Tasks with YAML

    main

    Tasks are defined in YAML files located in the config/ directory. A configuration file specifies which task classes to run, their schedule, and their specific parameters.

    Example configuration for a notebook-based data collection task:

    tasks:
      data_collection:
        enabled: true
        task_class: app.tasks.notebook.notebook_task.NotebookTask
        schedule:
          type: interval
          hours: 6
        config:
          notebooks:
            - data_collection/download_candles_all_pairs.ipynb
            - feature_engineering/trend_follower_grid.ipynb
          output_dir: app/outputs/cohort-12
  4. How the candle cache merging works

    main

    The CLOBDataSource cache system is designed to be additive and efficient. When you request data using methods like get_candles_last_days, the system performs the following logic:

    1. Instant Retrieval: If the requested range is already fully present in the cache, it returns the data immediately without network calls.
    2. Partial Fetching: If only a portion of the requested range is cached, the system identifies the missing time segments and only fetches the necessary data from the exchange.
    3. Automatic Merging: New data is automatically merged with existing Parquet-based cache files on disk.
    4. Storage: Data is stored in Parquet format to ensure efficient disk usage and fast I/O.
  5. How the improved feature engineering system works

    main

    The feature engineering system is designed for efficiency and type safety using Pydantic models. Key concepts include:

    • One feature per strategy: Instead of creating multiple documents for every individual indicator, a single Feature object contains all related calculation data (e.g., multiple EMA values) in a single dictionary. This significantly reduces database storage requirements.
    • Flexible Value Types: Feature values can be a float, List[float], or Dict[str, float].
    • Standardized Signal Model: Signals use a normalized scale from -1 (strong short) to 1 (strong long).
    • Storage: FeatureStorage is used for persistence, specifically optimized for MongoDB in production environments.
  6. Calculate position metrics for price range analysis

    main

    To analyze where the current price sits within a recent price range, use the following logic (often implemented via add_position_metrics):

    • max_price: Highest price in the lookback period.
    • min_price: Lowest price in the lookback period.
    • current_position: A value between 0 and 1, where 0 is at the period minimum and 1 is at the period maximum. Calculated as (max_price - current_price) / (max_price - min_price).
    • range_price_pct: The price range as a percentage of the current price: (max_price - current_price) / current_price.
  7. Understand the Smart Bot Orchestration Workflow

    main

    The Smart Bot Orchestration system automates grid trading by following a multi-step pipeline to manage bot lifecycles based on market signals and performance. The typical workflow consists of:

    1. Signal Analysis: Evaluating market signals to determine required trading directions.
    2. Check Active Bots: Auditing currently running controllers.
    3. Analyze & Stop Controllers: Identifying and stopping controllers that no longer meet performance or signal criteria.
    4. Archive Dead Bots: Cleaning up and archiving inactive bot instances.
    5. Deploy New Controllers: Launching new controllers based on fresh signals.
    6. Final Summary: Generating a performance and status report.
  8. Calculate buy/sell volume and net volume

    main

    Market volume analysis can be performed by deriving buy_volume and sell_volume from the candle data.

    1. Buy Volume: Use the taker_buy_base_volume field.
    2. Sell Volume: Subtract buy_volume from the total volume.
    3. Net Volume: buy_volume - sell_volume.
    4. Cumulative Net Volume: The running sum of net_volume over time.
    # Assuming 'candles.data' is a pandas DataFrame
    df["buy_volume"] = df["taker_buy_base_volume"]
    df["sell_volume"] = df["volume"] - df["buy_volume"]
    
    # Grouping by date for daily analysis
    grouped = df.groupby(df.index.date).agg({
        "open": "first",
        "high": "max",
        "low": "min",
        "close": "last",
        "volume": "sum",
        "buy_volume": "sum",
        "sell_volume": "sum",
    }).reset_index().rename(columns={"index": "date"})
    
    # Calculate net metrics
    grouped["net_volume"] = grouped["buy_volume"] - grouped["sell_volume"]
    grouped["cumulative_net"] = grouped["net_volume"].cumsum()
  9. Generate MACD and Bollinger Band trading signals

    main

    A trading signal can be generated by combining Bollinger Band position (relative to thresholds) with MACD momentum and direction. In this implementation, a long signal is triggered when the price is below a lower Bollinger Band threshold and MACD shows positive momentum while being below zero. A short signal is triggered when the price is above an upper threshold and MACD shows negative momentum while being above zero.

    # Define thresholds
    bb_long_threshold = 0
    bb_short_threshold = 1
    
    # Column names generated by pandas_ta follow the pattern: {indicator}_{params}
    # Example: BBP_100_1.5, MACDh_21_42_9, MACD_21_42_9
    
    candles_df["signal"] = 0
    bbp = candles_df[f"BBP_{bb_length}_{bb_std}"]
    macdh = candles_df[f"MACDh_{macd_fast}_{macd_slow}_{macd_signal}"]
    macd = candles_df[f"MACD_{macd_fast}_{macd_slow}_{macd_signal}"]
    
    # Long: BB Percent < threshold AND MACD Histogram > 0 AND MACD < 0
    long_condition = (bbp < bb_long_threshold) & (macdh > 0) & (macd < 0)
    
    # Short: BB Percent > threshold AND MACD Histogram < 0 AND MACD > 0
    short_condition = (bbp > bb_short_threshold) & (macdh < 0) & (macd > 0)
    
    candles_df.loc[long_condition, "signal"] = 1
    candles_df.loc[short_condition, "signal"] = -1
  10. Implement Triple EMA Trend Following strategy

    main

    The Triple EMA strategy identifies trends by requiring three Exponential Moving Averages (EMA) to align in a specific hierarchy. This reduces false signals by ensuring momentum is consistent across short, medium, and long timeframes.

    EMA Configuration

    • Fast EMA (e.g., 12): Captures short-term momentum.
    • Medium EMA (e.g., 100): Filters noise and confirms the trend.
    • Slow EMA (e.g., 300): Identifies the major trend direction.

    Signal Logic

    • Bullish (Buy) Signal: Fast EMA > Medium EMA > Slow EMA (all aligned upward).
    • Bearish (Sell) Signal: Fast EMA < Medium EMA < Slow EMA (all aligned downward).

    Implementation Example

    Use the pandas_ta library to append EMAs to your dataframe, then apply the conditional logic:

    import pandas_ta as ta
    
    # Setup periods
    ema_short, ema_medium, ema_long = 12, 100, 300
    
    # Calculate EMAs
    df.ta.ema(length=ema_short, append=True)
    df.ta.ema(length=ema_medium, append=True)
    df.ta.ema(length=ema_long, append=True)
    
    # Extract series
    short_ema = df[f"EMA_{ema_short}"]
    medium_ema = df[f"EMA_{ema_medium}"]
    long_ema = df[f"EMA_{ema_long}"]
    
    # Generate signals
    long_condition = (short_ema > medium_ema) & (medium_ema > long_ema) & (short_ema > long_ema)
    short_condition = (short_ema < medium_ema) & (medium_ema < long_ema) & (short_ema < long_ema)
    
    df["signal"] = 0
    df.loc[long_condition, "signal"] = 1
    df.loc[short_condition, "signal"] = -1
    import pandas_ta as ta
    
    # Calculate EMAs
    df.ta.ema(length=12, append=True)
    df.ta.ema(length=100, append=True)
    df.ta.ema(length=300, append=True)
    
    # Signal logic
    long_condition = (df['EMA_12'] > df['EMA_100']) & (df['EMA_100'] > df['EMA_300'])
    short_condition = (df['EMA_12'] < df['EMA_100']) & (df['EMA_100'] < df['EMA_300'])
    
    df['signal'] = 0
    df.loc[long_condition, 'signal'] = 1
    df.loc[short_condition, 'signal'] = -1
  11. Manage rate limits for candle downloads

    main

    Because Hummingbot candles do not currently use a shared rate limit, you must manually configure batching and sleep intervals to avoid 429 (Too Many Requests) errors.

    Recommended Safety Settings:

    • BATCH_CANDLES_REQUEST: Keep this low (between 2 and 3) to minimize the number of simultaneous requests.
    • SLEEP_REQUEST: Increase this to 10+ seconds between batches to ensure compliance with exchange limits.
  12. Calculate Uniswap V3 price from sqrtPriceX96

    main

    Uniswap V3 uses sqrtPriceX96 to represent price. To convert this to a human-readable price between token0 and token1:

    1. Calculate the raw price: price = (sqrt_price_x96 / 2^96)^2.
    2. Adjust for token decimals: price_adjusted = price * (10^token0_decimals) / (10^token1_decimals).

    This gives the price of token1 expressed in units of token0.

    # slot0[0] is sqrtPriceX96
    sqrt_price = slot0[0] / (2 ** 96)
    price = sqrt_price ** 2
    price_adjusted = price * (10 ** token0_decimals) / (10 ** token1_decimals)