Prediction Market Analysis

The project is organized as follows:

• src/analysis/: Contains analysis scripts for kalshi/ and polymarket/.
• src/indexers/: Contains data collection indexers for kalshi/ (API client) and polymarket/ (API/blockchain).
• src/common/: Shared utilities and interfaces.
• data/: The storage directory for markets, trades, and blocks (Kalshi and Polymarket).
• docs/: Project documentation.
• output/: Generated analysis outputs (figures, CSVs).

New analysis scripts should be placed in src/analysis/{kalshi,polymarket}/ and must extend the Analysis base class.

An analysis class requires:

1. name: A string identifier.
2. description: A brief summary of the analysis.
3. run(): A method that returns a tuple containing a matplotlib.figure.Figure and a dict of data (used for CSV/JSON export).

from pathlib import Path
import duckdb
import matplotlib.pyplot as plt
from matplotlib.figure import Figure
from src.common.analysis import Analysis

class MyAnalysis(Analysis):
    name = "my_analysis"
    description = "Brief description of what this analysis does"

    def run(self) -> tuple[Figure, dict]:
        # ... implementation ...
        return fig, df.to_dict(orient="records")

Execute the analysis framework to generate figures and statistics. Running this command opens an interactive menu where you can choose to run all available analyses or select a specific one. Output files, including PNG, PDF, CSV, and JSON, are saved to the output/ directory.

make analyze

Run the indexing command to gather new data from prediction market APIs. This command opens an interactive menu allowing you to select specific indexers. Data is stored in data/kalshi/ and data/polymarket/. The process supports automatic progress saving, allowing you to interrupt and resume collection later.

make index

To create a new analysis, subclass Analysis and implement the run() method. The run() method must return an AnalysisOutput object containing your results (figures, dataframes, or chart configurations). You can then use the save() method to export these results to various file formats.

from common.analysis import Analysis, AnalysisOutput
import matplotlib.pyplot as plt
import pandas as pd

class MyAnalysis(Analysis):
    def run(self) -> AnalysisOutput:
        # Generate your figure and data
        fig, ax = plt.subplots()
        ax.plot([1, 2, 3], [1, 4, 9])
        df = pd.DataFrame({"x": [1, 2, 3], "y": [1, 4, 9]})

        return AnalysisOutput(
            figure=fig,
            data=df,
            # chart=line_chart(...) # optional
        )

# Instantiate and save
analysis = MyAnalysis("my_analysis", "A simple quadratic plot")
analysis.save("output/")

Use the progress() context manager within your run() method to display a spinner in the CLI during expensive data loading or computation steps.

def run(self) -> AnalysisOutput:
    with self.progress("Loading trades data"):
        df = con.execute("SELECT * FROM large_table").df()

    with self.progress("Computing aggregations"):
        # expensive computation
        result = df.groupby(...).agg(...)

The src.analysis.kalshi.util.categories module provides utilities for mapping Kalshi event tickers to human-readable hierarchies and consistent visualization colors.

• get_group(ticker): Returns the high-level group (e.g., 'Sports').
• get_hierarchy(ticker): Returns a tuple of (group, category, subcategory).
• GROUP_COLORS: A dictionary mapping groups to hex color codes.

from src.analysis.kalshi.util.categories import get_group, get_hierarchy, GROUP_COLORS

# Get high-level group
group = get_group("NFLGAME")  # Returns "Sports"

# Get full hierarchy (group, category, subcategory)
hierarchy = get_hierarchy("NFLGAME")  # Returns ("Sports", "NFL", "Games")

# Use predefined colors for consistent visualizations
color = GROUP_COLORS["Sports"]  # Returns "#1f77b4"

When initializing WinRateByTradeSizeAnalysis, you can specify the directories containing your Kalshi data. If not provided, the class defaults to a relative path based on the project root: data/kalshi/trades and data/kalshi/markets.

• trades_dir (Path | str | None): Path to the directory containing Kalshi trade parquet files.
• markets_dir (Path | str | None): Path to the directory containing Kalshi market parquet files.

The PolymarketLegacyTradesIndexer uses specific paths for data storage and progress tracking:

• Trade Data Directory: data/polymarket/legacy_trades (stores .parquet files).
• Progress Cursor: data/polymarket/.legacy_backfill_block_cursor (stores the last processed block to allow resuming).

Note: When saving to Parquet, large integer fields (amount, fee_amount, and outcome_tokens) are converted to strings to prevent overflow issues.