StockBench

The storage/ directory is the central repository for the Trading Agent system, containing cached API data, partitioned market data, execution logs, and backtest reports.

Directory Overview:

• cache/: Cached data from external APIs (corporate actions, financials, LLM responses, news, stock indicators).
• logs/: System execution logs for debugging.
• parquet/: Partitioned market data (OHLCV) stored in Parquet format.
• reports/: Generated backtest results, including trades, NAV, metrics, and visualizations.

To run backtests or access market data, you must set the following environment variables. If you wish to test specific months or stocks, ensure these keys are configured.

• POLYGON_API_KEY: Required for high-quality price and fundamental data.
• FINNHUB_API_KEY: Required for timely news data.
• OPENAI_API_KEY: Required for LLM-powered decision making.

Free tiers are available at Polygon.io and Finnhub.io.

export POLYGON_API_KEY="your_polygon_api_key"
export FINNHUB_API_KEY="your_finnhub_api_key"
export OPENAI_API_KEY="your_openai_api_key"

Follow these steps to set up the StockBench environment using conda and pip.

1. Clone the repository.
2. Create a new Conda environment with Python 3.11.
3. Activate the environment.
4. Install the required dependencies via requirements.txt.

# Clone the repository
git clone <repository-url>
cd stockbench

# Create environment
conda create -n stockbench python=3.11
conda activate stockbench

# Install dependencies
pip install -r requirements.txt

The pre-cache utility uses a _safe_api_call wrapper to handle API instability and rate limits:

1. Standard Delay: An api_delay is applied before every call to maintain a steady request rate.
2. Rate Limit (429) Handling: If an error containing 429, api limit, or rate limit is detected, the script pauses for the duration of batch_delay before retrying.
3. Network Error Handling: For timeout or network errors, the script uses exponential backoff (up to a maximum of 30 seconds) for retries.
4. Immediate Failures: Errors like authentication failures do not trigger retries and cause the specific data fetch to fail immediately.

The run_backtest.py script is the primary CLI entrypoint for executing LLM-powered stock trading backtests. It requires a configuration file (YAML) and allows overriding various parameters such as date ranges, symbols, risk controls, and benchmark settings via command-line flags.

If no symbols are provided via the --symbols flag, the tool defaults to the symbols_universe defined in your configuration file.

Key Workflow:

1. Prepare a YAML configuration file.
2. Execute the script using python run_backtest.py with the required --cfg path.
3. Optionally override settings for specific runs (e.g., changing the date range or symbols).
4. Locate the results in the output directory printed at the end of the execution.

python stockbench/apps/run_backtest.py --cfg config.yaml --start 2024-01-01 --end 2024-12-31 --symbols AAPL,MSFT

When working with the storage directory, address common issues as follows:

• Missing Data: Verify if the requested symbol exists in the parquet/ directory.
• Corrupted Files: Check the corresponding .sha256 files to verify data integrity.
• Permission Errors: Ensure the application has write access to the storage/ subdirectories.
• Disk Space: Monitor usage, as large datasets (especially news and minute-level parquet) can consume significant space.
• Validation: Check logs/{RUN_ID}.log for data ingestion errors and validate Parquet file schemas before processing.

Each backtest run generates a unique directory in storage/reports/backtest/{RUN_ID}/.

Key Files in a Run Directory:

• trades.parquet: Individual trade records (timestamps, symbols, quantities, prices).
• daily_nav.parquet: Daily net asset value (NAV) progression.
• metrics.json: Comprehensive performance metrics.
• metrics_summary.csv: CSV version of performance metrics.
• conclusion.md: Detailed analysis and insights.
• config.json: Snapshot of the configuration used for the run.

import pandas as pd
import json

# Load trade data
trades = pd.read_parquet('storage/reports/backtest/EXP_OPENAI_MAR_7/trades.parquet')

# Load daily NAV
nav = pd.read_parquet('storage/reports/backtest/EXP_OPENAI_MAR_7/daily_nav.parquet')

# Load comprehensive metrics
with open('storage/reports/backtest/EXP_OPENAI_MAR_7/metrics.json', 'r') as f:
    metrics = json.load(f)

Market data is stored in partitioned Parquet format within storage/parquet/{SYMBOL}/{granularity}/. Granularities include day/ and minute/. Each .parquet file is accompanied by a .sha256 checksum for integrity verification.

Supported Symbols include: AAPL, MSFT, GOOGL, AMZN, NVDA, TSLA, META, JPM, GS, HD, UNH, V, JNJ, PG, MCD, IBM, HON, CAT, BA, AXP, CRM, TRV, SHW, AMGN, and the ETF SPY.

import pandas as pd

# Load daily data for AAPL
df = pd.read_parquet('storage/parquet/AAPL/day/2025-01-01.parquet')

Fundamental and sentiment data are cached in JSON format:

• Financials: Located in storage/cache/financials/. Files include {SYMBOL}.all.json (complete dataset), {SYMBOL}.annual.json, and {SYMBOL}.quarterly.json.
• News: Located in storage/cache/news/ (general articles) and storage/cache/news_by_day/{SYMBOL}/ (daily organized news by symbol).

# Load complete financial dataset
with open('storage/cache/financials/AAPL.all.json', 'r') as f:
    all_financials = json.load(f)

# Load quarterly financials only
with open('storage/cache/financials/AAPL.quarterly.json', 'r') as f:
    quarterly_financials = json.load(f)

# Load daily news for AAPL
with open('storage/cache/news_by_day/AAPL/2024-12-01.json', 'r') as f:
    daily_news = json.load(f)

The run_backtest_main function is exported for programmatic access to the backtesting application. This allows you to trigger the backtest runner directly from Python code instead of using the CLI.

from stockbench.apps import run_backtest_main

# Note: This typically expects CLI-style arguments or environment configuration
run_backtest_main()

Use the list_applications() function to retrieve a list of all available application names within the stockbench.apps package. This is useful for programmatic discovery of available CLI tools.

from stockbench.apps import list_applications

apps = list_applications()
print(apps)  # Output: ['run_backtest']