zipline-reloaded Documentation

repository·main·Indexed 23 days ago

https://github.com/stefan-jansen/zipline-reloaded

A backtesting engine for trading strategies that allows developers to simulate algorithms using historical data. It features a Trading Algorithm API for managing orders, risk controls, and simulation parameters like commission and slippage. The library includes a Pipeline API for efficient factor computation, a Data API for managing market data bundles, and tools for monitoring algorithm state and risk metrics.

Tokens
17.7K
Snippets
39
Records
103
Agent score
78%

What's inside zipline-reloaded

  1. What is a Trading Calendar and why is it used?

    main

    A TradingCalendar represents the timing information of a single market exchange. It consists of two main components:

    1. Sessions: A contiguous set of minutes with a label (conventionally midnight UTC).
    2. Opens/Closes: The specific times the market starts and ends trading.

    Importance for Backtesting: Your TradingCalendar must match the dates available in your data bundle. If you attempt to place a trade on a day when the exchange is closed (e.g., a Saturday for the NYSE), the backtest will fail or produce errors. This applies to both minutely and daily data.

  2. How to define a custom metric

    main

    A metric is an object that implements one or more of the following lifecycle methods to collect and report data. If a metric does not need to process data at a specific stage, the method can be omitted.

    Lifecycle Methods

    1. start_of_simulation(self, ledger, emission_rate, trading_calendar, sessions, benchmark_source) Acts as a per-simulation constructor. Use this to initialize caches.

      • emission_rate: A string ('minute' or 'daily'). If 'daily', end_of_bar is not called.
    2. end_of_simulation(self, packet, ledger, trading_calendar, sessions, data_portal, benchmark_source) Used to write final values into the packet dictionary.

    3. start_of_session(self, ledger, session_label, data_portal) Called at the start of a session. Useful if prices or capital change between sessions.

    4. end_of_session(self, packet, ledger, session_label, session_ix, data_portal) Called at the end of a session. The packet dictionary contains daily_perf and cumulative_perf sub-dictionaries for writing values.

    5. end_of_bar(self, packet, ledger, dt, session_ix, data_portal) Note: Only called when emission_rate is 'minute'. The packet dictionary contains minute_perf and cumulative_perf sub-dictionaries.

  3. Understand the Pipeline Engine execution flow

    main

    The SimplePipelineEngine executes pipelines using the following steps:

    1. Domain Determination: Determine the domain of the pipeline.
    2. Dependency Graph: Build a graph of all terms and their required extra rows.
    3. Lifetimes Matrix: Combine domain with AssetFinder to create a (date x asset) boolean matrix of tradability.
    4. Workspace Creation: Produce a dictionary of cached/pre-computed terms.
    5. Topological Sort: Sort terms to determine execution order.
    6. Iterative Computation: For each term, fetch inputs from the workspace, compute the term, store it, and prune unused results to save memory.
    7. Output Extraction: Convert workspace results into a 'narrow' format based on the pipeline's screen.
  4. Use the Pipeline API for efficient factor computation

    main

    The Pipeline API enables faster and more memory-efficient execution by optimizing the computation of factors during a backtest. It allows you to define a set of factors and filters to screen for assets.

    Core components include:

    • Pipeline: The main container for factors and filters.
    • Factor: Used to transform input data into signals (e.g., rank, zscore, mean, stddev).
    • Filter: Used to screen assets (e.g., All, Any, AtLeastN).
    • CustomFactor: For user-defined factor logic.
    • DataSet and Column: To access underlying data.

    To use a pipeline in an algorithm, use attach_pipeline(pipeline) and retrieve results via pipeline_output().

  5. Implement the ingest function: Writer arguments

    main

    When implementing a custom ingest function, you are provided with several writer objects to convert your data into Zipline's internal format:

    • asset_db_writer (AssetDBWriter): Writes asset metadata (lifetimes, symbol to SID mapping, name, exchange). Use .write(df).
    • minute_bar_writer (BcolzMinuteBarWriter): Writes minute-level data. Use .write(iterable_of_tuples) where tuples are (sid, dataframe). SIDs can repeat if dates are strictly increasing. Forward show_progress to this method.
    • daily_bar_writer (BcolzDailyBarWriter): Writes daily-level data. Use .write(iterable_of_tuples) where tuples are (sid, dataframe). Unlike minute bars, a SID should only appear once in the iterable. Forward show_progress to this method.
    • adjustment_writer (SQLiteAdjustmentWriter): Writes splits, mergers, and dividends. Use .write(df).
    • cache (dataframe_cache): A mapping from strings to dataframes. Use this to store raw data during ingestion to prevent re-downloading if the process crashes. The cache is only cleared upon successful completion.
  6. Requirements for using custom calendars with data bundles

    main
    When using a custom TradingCalendar, ensure your data bundle contains asset data that covers the specific trading days defined in your calendar. For example, if your calendar defines a 24/7 trading schedule (including weekends), your data bundle must include data for Saturdays and Sundays. If the data is missing for those days, the backtest will encounter errors.
  7. How to construct a Zipline algorithm

    main

    A Zipline algorithm is defined by two mandatory functions that manage the lifecycle of a backtest:

    1. initialize(context): Called once before the simulation starts. Use the context object as a persistent namespace to store variables that need to be accessed across different algorithm iterations (e.g., state, parameters, or indicators).
    2. handle_data(context, data): Called repeatedly for each event (e.g., every trading day). It receives the context namespace and a data event-frame. The data object contains the current trading bar (OHLC prices and volume) for each security in your universe.

    All core trading functions are imported from zipline.api.

    from zipline.api import order, record, symbol
    
    
    def initialize(context):
        # Initialize state or parameters here
        pass
    
    
    def handle_data(context, data):
        # Trading logic goes here
        pass
  8. Run algorithms in Jupyter Notebook using `%%zipline` magic

    main

    To run an algorithm directly within a Jupyter Notebook cell, first load the Zipline extension using %load_ext zipline. Then, use the %%zipline magic command at the top of your cell. This magic command accepts the same arguments as the CLI. When using the magic, you do not need to specify an input file, and you can assign the resulting performance DataFrame to a variable using the -o flag.

    %load_ext zipline
    
    %%zipline --start 2016-1-1 --end 2018-1-1 -o perf_df
    from zipline.api import symbol, order, record
    
    def initialize(context):
        pass
    
    def handle_data(context, data):
        order(symbol('AAPL'), 10)
        record(AAPL=data.current(symbol('AAPL'), "price"))
  9. How to build a custom TradingCalendar

    main

    To create a custom exchange calendar, you must inherit from the zipline.utils.calendars.trading_calendar.TradingCalendar class and implement several required properties.

    Key properties to implement:

    • name: The identifier for the exchange (used with the --trading-calendar CLI flag).
    • tz: The timezone for the exchange (using pytz).
    • open_time: The daily opening time (using datetime.time).
    • close_time: The daily closing time (using datetime.time).
    • regular_holidays: A pandas.tseries.holiday.HolidayCalendar object defining recurring holidays.
    • day (optional): Use @lazyval with pandas.tseries.offsets.CustomBusinessDay to define which days of the week the exchange is open (e.g., via a weekmask).
    from datetime import time
    import pandas as pd
    from pandas.tseries.offsets import CustomBusinessDay
    from pytz import timezone
    from zipline.utils.calendar_utils import register_calendar, TradingCalendar
    from zipline.utils.memoize import lazyval
    
    class TFSExchangeCalendar(TradingCalendar):
        """
        An exchange calendar for trading assets 24/7.
    
        Open Time: 12AM, UTC
        Close Time: 11:59PM, UTC
        """
    
        @property
        def name(self):
          return "TFS"
    
        @property
        def tz(self):
          return timezone("UTC")
    
        @property
        def open_time(self):
          return time(0, 0)
    
        @property
        def close_time(self):
          return time(23, 59)
    
        @lazyval
        def day(self):
          weekmask = "Mon Tue Wed Thu Fri Sat Sun"
          return CustomBusinessDay(
            weekmask=weekmask
          )