DOCSEARCHbyAlgolia
Submit Docs

PokieTicker Documentation

repository·main·Indexed 21 days ago

https://github.com/owengetinfo-design/pokieticker

An event-driven stock analysis tool that visualizes the relationship between news events and price movements. It utilizes Claude AI for news sentiment scoring and an XGBoost classifier to predict short-term price trends based on 31 engineered news and technical features. The system features a React/D3.js frontend and a FastAPI/SQLite backend with integration for the Polygon.io API.

Tokens
2.6K
Snippets
9
Records
12
Agent score
73%

What's inside PokieTicker

  1. Overview of PokieTicker Architecture and API

    main

    PokieTicker is a full-stack application consisting of a React/D3.js frontend and a FastAPI/SQLite backend. It uses an XGBoost-based prediction system that combines news sentiment with technical indicators.

    Backend API Endpoints

    The backend exposes several key API routes used by the frontend:

    • /api/stocks/{sym}/ohlc: Retrieves OHLC price data for a specific symbol.
    • /api/news/{sym}?date=: Retrieves news for a specific symbol and date.
    • /api/news/{sym}/categories: Retrieves news categorized by impact type.
    • /api/predict/{sym}/forecast: Generates 7-day and 30-day forecasts.
    • /api/analysis/deep: Triggers deep AI analysis (Sonnet) on a specific story or range.
    • /api/pipeline/fetch & /api/pipeline/process: Triggers data ingestion and processing pipelines.

    Data Pipeline Layers

    The system processes data through three distinct layers:

    1. Layer 0 (Rule Filter): Filters raw Polygon API data to reject spam and listicles (~17% rejection rate).
    2. Layer 1 (Haiku Batch API): Uses Claude Haiku to score news sentiment and extract bullish/bearish reasons in batches.
    3. Layer 2 (Sonnet On-Demand): Uses Claude Sonnet for deep, detailed analysis when a user clicks a specific news item.

  2. How the XGBoost prediction system works

    main

    The prediction model uses an XGBoost classifier to predict price direction at T+1, T+3, and T+5 horizons. It relies on 31 engineered features:

    News Features:

    • Article count
    • Sentiment score
    • Positive/negative ratio
    • 3/5/10-day rolling averages
    • Sentiment momentum

    Technical Features:

    • Price returns (1/3/5/10-day)
    • Volatility
    • Volume ratio
    • RSI-14
    • Moving average crossover

    Pattern Matching: The system also uses cosine similarity on these feature vectors to find historically similar news patterns and show what happened to the stock following those events.

  3. Quick Start: Run PokieTicker locally

    main

    You can run PokieTicker immediately using the pre-built database and models, which requires no API keys. Follow these steps to set up the backend and frontend environments.

    1. Clone and Unpack Data

    git clone https://github.com/owengetinfo-design/PokieTicker.git
cd PokieTicker

# Unpack the pre-built database and models
gunzip -k pokieticker.db.gz
tar xzf models.tar.gz -C backend/ml/

    2. Setup Backend (Python 3.10+)

    python -m venv venv
source venv/bin/activate   # Windows: venv\Scripts\activate
pip install -r requirements.txt

    3. Setup Frontend (Node.js 18+)

    cd frontend && npm install && cd ..

    4. Run the Application

    Open two terminal windows and run:

    Terminal 1: Backend

    source venv/bin/activate
uvicorn backend.api.main:app --reload

    Terminal 2: Frontend

    cd frontend && npm run dev

    Once started, access the app at http://localhost:7777/PokieTicker/.

    # Terminal 1: Backend
source venv/bin/activate
uvicorn backend.api.main:app --reload

# Terminal 2: Frontend
cd frontend && npm run dev

  4. Update stock data and run AI analysis

    main

    If you have configured your .env file with API keys, you can manually trigger data ingestion and AI processing.

    Fetching New Data

    To fetch new OHLC (Open, High, Low, Close) and news data:

    python -m backend.bulk_fetch

    Running AI Analysis

    To process news articles using the Anthropic Batch API:

    1. Submit for batch processing (processes the top 50 tickers):
      python -m backend.batch_submit --top 50
    2. Collect results (replace <batch_id> with the ID returned from the submission):
      python -m backend.batch_collect <batch_id>

    Weekly Incremental Updates

    To perform a routine update of news and OHLC data since the last run:

    python -m backend.weekly_update
python -m backend.batch_submit --top 50
python -m backend.batch_collect <batch_id>
    # Fetch new OHLC + news
python -m backend.bulk_fetch

# Run AI analysis
python -m backend.batch_submit --top 50
python -m backend.batch_collect <batch_id>

  5. Configure API keys for data updates

    main

    To fetch the latest stock data and perform AI analysis beyond the pre-built dataset, you must provide your own API keys in a .env file.

    1. Copy the example environment file: cp .env.example .env
    2. Edit .env and populate the following keys:
    KeyProviderDescription
    POLYGON_API_KEYpolygon.ioRequired for OHLC and news data.
    ANTHROPIC_API_KEYAnthropicRequired for Claude Haiku (batch) and Sonnet (on-demand) analysis.
    cp .env.example .env

  7. Fetch OHLC data with fetch_ohlc()

    main

    Use fetch_ohlc to retrieve daily Open, High, Low, Close (OHLC) data for a specific ticker within a date range. The function handles the Polygon API request and transforms the raw response into a standardized list of dictionaries.

    Returns a list of dictionaries with the following keys:

    • date: ISO format date string.
    • open: Opening price.
    • high: Highest price.
    • low: Lowest price.
    • close: Closing price.
    • volume: Trading volume.
    • vwap: Volume Weighted Average Price.
    • transactions: Number of transactions.
    from backend.polygon.client import fetch_ohlc

# Fetch daily data for AAPL from 2023-01-01 to 2023-01-10
ohlc_data = fetch_ohlc("AAPL", "2023-01-01", "2023-01-10")

for day in ohlc_data:
    print(f"{day['date']}: Close {day['close']}")

  8. Internal HTTP utility: http_get()

    main

    The http_get function is an internal utility used by the client methods to perform authenticated GET requests to Polygon. It implements robust error handling including:

    • Exponential Backoff: Retries on connection exceptions.
    • 429 Handling: Respects the Retry-After header or uses a calculated backoff when rate-limited.
    • 5xx Handling: Retries on server-side errors.
    • Authentication: Automatically injects the Authorization: Bearer <API_KEY> header using settings.polygon_api_key.

  9. Fetch news articles with fetch_news()

    main

    Use fetch_news to retrieve news articles related to a specific ticker within a date range. The function automatically handles pagination to collect multiple pages of results.

    Parameters:

    • ticker (str): The stock ticker symbol.
    • start (str): Start date (ISO format).
    • end (str): End date (ISO format).
    • per_page (int, default 50): Number of results per page.
    • page_sleep (float, default 1.2): Seconds to sleep between paginated requests to avoid rate limiting.
    • max_pages (Optional[int]): Limit the total number of pages fetched.

    Returns a list of dictionaries with the following keys:

    • id: Unique article ID.
    • publisher: Name of the news publisher.
    • title: Article title.
    • author: Author name.
    • published_utc: Publication timestamp.
    • amp_url: AMP version of the URL.
    • article_url: Original article URL.
    • tickers: List of tickers mentioned.
    • description: Article summary/description.
    • insights: AI-generated or provided insights.
    from backend.polygon.client import fetch_news

# Fetch news for TSLA in January 2024
news = fetch_news("TSLA", "2024-01-01", "2024-01-31", max_pages=3)

for article in news:
    print(f"[{article['published_utc']}] {article['title']}")

  10. Search for tickers with search_tickers()

    main

    Use search_tickers to find ticker symbols, names, and sectors using a search query against the Polygon reference endpoint. This is useful for validating ticker symbols or finding related stocks.

    Parameters:

    • query (str): The search string (e.g., "Apple").
    • limit (int, default 20): Maximum number of results to return.

    Returns a list of dictionaries with the following keys:

    • symbol: The ticker symbol.
    • name: The company name.
    • sector: The industry sector (from sic_description).
    from backend.polygon.client import search_tickers

# Search for companies related to 'Microsoft'
results = search_tickers("Microsoft", limit=5)

for r in results:
    print(f"{r['symbol']} - {r['name']} ({r['sector']})")

  11. Initialize the PokieTicker backend service

    main

    The PokieTicker backend is a FastAPI application. Upon startup, the application automatically triggers the init_db() function to ensure the database is initialized. The service exposes several API routers organized by domain: stocks, news, analysis, and predict.

    from fastapi import FastAPI

app = FastAPI(title="PokieTicker", version="1.0.0")

# Routers are mounted at:
# /api/stocks
# /api/news
# /api/analysis
# /api/predict