Stocksera Documentation

repository·master·Indexed 21 days ago

https://github.com/guanquann/stocksera

A financial analysis platform that aggregates data from Yahoo Finance, SEC, Reddit, and other sources. It provides insights into stock tickers, options, social sentiment, and macro-economic indicators via a web dashboard and a set of scheduled Python tasks. Features include ticker analysis, options chains, short interest tracking, and government trading data.

Tokens
2.4K
Snippets
6
Records
12
Agent score
68%

What's inside Stocksera

  1. Overview of Stocksera Web Dashboard Endpoints

    master

    The Stocksera web application provides various endpoints for financial analysis. Key features include:

    • Ticker Analysis (/ticker/): View graphs, key statistics (EPS, beta, SMA), historical data, insider trading, news sentiment, Google trends, and recommendations.
    • Options (/ticker/options/): View options chains, max-pain price, OTM/ITM options, and Call/Put ratios.
    • Short Interest & Volume: View short volume (/ticker/short_volume/), failure to deliver (/ticker/failure_to_deliver/), borrowed shares (/ticker/borrowed_shares/), and Regulation SHO Threshold List (/ticker/regsho/).
    • Social Sentiment: Real-time tracking of Reddit (WSB, r/stockmarket, r/stocks, r/CryptoCurrency) via /wsb_live/, /crypto_live/, /reddit_analysis/, and /subreddit_count/.
    • Market Data: Market summaries (/market_summary/), futures (/futures/), earnings calendars (/earnings_calendar/), and IPO calendars (/ipo_calendar/).
    • Macro & Government: Senate/House trading (/senate/, /house/), inflation rates (/inflation/), retail sales (/retail_sales/), and treasury data (/daily_treasury/).
  2. Set up Stocksera with Docker

    master

    To run Stocksera using Docker, follow these steps:

    1. Configure Database: Set your MYSQL credentials in config.yaml first. Note that for Docker containers, MYSQL_HOST might not be localhost (e.g., it might be db).
    2. Run Docker Setup:
      py setup/docker.py
    3. **Initialize Database**: If setting up for the first time, run the database setup script in a separate command prompt:
       ```bash
    py setup/docker_db.py

    Note: You may need to rerun setup/docker.py even if it shows errors, and you might need to rerun it after setup/docker_db.py completes successfully. Always ensure API keys are configured at http://localhost:8000/setup/.

    py setup/docker.py
    py setup/docker_db.py
  3. Run scheduled tasks in Stocksera

    master

    Scheduled tasks can be executed using two different methods:

    1. Web Interface: Navigate to http://localhost:8000/tasks and click the buttons corresponding to the tasks you wish to run.
    2. Command Prompt: Run the Python scripts directly from the terminal.

    Important Requirements:

    • All commands must be executed from the main parent directory of the project.
    • You must run scheduled_tasks/create_database.py before running any other scheduled tasks.
    • Ensure all necessary API keys are configured in config.yaml or via the web interface at http://localhost:8000/tasks to prevent errors.
    # Example: Running a task from the main parent directory
    # Ensure you are in the project root (e.g., C:\Users\Desktop\Stocksera)
    python scheduled_tasks/economy/get_inflation.py
  4. Configure API keys for scheduled tasks

    master

    To prevent errors during task execution, you must register for APIs from the following providers and add the credentials to either config.yaml or the web interface at http://localhost:8000/tasks:

    API SourceRegistration URL
    Reddithttps://www.reddit.com/prefs/apps
    Twitterhttps://developer.twitter.com/en/portal/dashboard
    Finnhubhttps://finnhub.io/
    Polygonhttps://polygon.io/
    FMPhttps://site.financialmodelingprep.com/
  5. Set up Stocksera locally (without Docker)

    master

    Follow these steps to set up a local development environment for Stocksera:

    1. Create and activate a virtual environment:
      py -m venv venv
      cd venv/Scripts
      activate
      cd ../..
    2. **Install dependencies**:
       ```bash
    pip install -r requirements.txt
    1. Configure Database: You must set your MYSQL credentials in config.yaml before proceeding. Then run the local setup script:
      py setup/local.py
    4. **Run the application**:
       ```bash
    py manage.py runserver

    CRITICAL: Ensure you have all necessary API keys configured at http://localhost:8000/setup/ to prevent errors during runtime.

    py -m venv venv
    cd venv/Scripts
    activate
    cd ../..
    pip install -r requirements.txt
    py setup/local.py
    py manage.py runserver
  6. Configure custom sentiment scores in custom_words.json

    master
    Use custom_extensions/custom_words.json to define custom sentiment scores for stock-specific terminology. This allows you to override standard linguistic sentiment; for example, you can map words like "bull" to a positive sentiment score, even though they might be neutral in standard English.
  7. Run Stocksera using Docker Compose

    master

    You can deploy the entire Stocksera stack (web application and MySQL database) using Docker Compose. The setup includes a web service running the Django application and a db service running MySQL 8.

    Service Details

    Web Service

    • Command: python manage.py runserver 0.0.0.0:8000
    • Port Mapping: Host 8000 maps to container 8000.
    • Volume: The current directory is mounted to /code inside the container.

    Database Service (db)

    • Image: mysql:8
    • Port Mapping: Host 3308 maps to container 3306.
    • Initialization: Any SQL scripts located in the local ./db directory are mounted to /docker-entrypoint-initdb.d/ as read-only and will be executed during database initialization.

    Environment Variables

    Both services use the following environment variables for MySQL configuration. Ensure these match your requirements:

    • MYSQL_ROOT_PASSWORD: The root password for the MySQL instance.
    • MYSQL_DATABASE: The name of the database to create (defaults to stocksera).
    • MYSQL_PASSWORD: The password for the application user.
    # To run the stack:
    docker-compose up
  8. Configure MySQL environment variables in Docker

    master

    When using the docker-compose.yml configuration, you can manage MySQL credentials and database names via environment variables in both the web and db services.

    VariableDescription
    MYSQL_ROOT_PASSWORDSets the MySQL root user password.
    MYSQL_DATABASESets the name of the database to be created on startup.
    MYSQL_PASSWORDSets the password for the database user.

    Note: Ensure that the credentials provided to the web service match those provided to the db service to allow the application to connect successfully.

    environment:
      - MYSQL_ROOT_PASSWORD=admin
      - MYSQL_DATABASE=stocksera
      - MYSQL_PASSWORD=admin
  9. Reference of scheduled task scripts and frequencies

    master

    The following table lists the available scheduled task scripts, their primary functions, and recommended execution frequencies. Note that frequencies are provided for guidance only and can be adjusted based on preference.

    Script NameFunctionsFrequency
    scheduled_tasks/reddit/stocks/scrape_trending_posts.pymain()Daily (PM)
    scheduled_tasks/reddit/stocks/scrape_discussion_thread.pywsb_live(), wsb_change(), get_mkt_cap(), update_hourly()10 Mins / Hourly
    scheduled_tasks/reddit/crypto/scrape_trending_posts.pymain()Daily (PM)
    scheduled_tasks/reddit/crypto/scrape_discussion_thread.pycrypto_live(), crypto_change(), update_hourly()10 Mins / Hourly
    scheduled_tasks/reddit/get_subreddit_count.pymain()Daily (MH)
    scheduled_tasks/twitter/get_twitter_followers.pymain()Daily
    scheduled_tasks/twitter/scrape_trending_posts.pymain()Daily (MH)
    scheduled_tasks/economy/get_reverse_repo.pyreverse_repo()1.30PM
    scheduled_tasks/economy/get_daily_treasury.pydownload_json()4.00PM
    scheduled_tasks/economy/get_inflation.pyinflation()Monthly
    scheduled_tasks/economy/get_retail_sales.pyretail_sales()Monthly
    scheduled_tasks/economy/get_interest_rate.pyinterest_rate()Monthly
    scheduled_tasks/economy/get_initial_jobless_claims.pyjobless_claims()Weekly
    scheduled_tasks/economy/get_upcoming_events_date.pymain()6.00PM
    scheduled_tasks/stocks/get_failure_to_deliver.pymain()2 Weeks
    scheduled_tasks/stocks/get_short_volume.pymain()6.00PM
    scheduled_tasks/stocks/get_borrowed_shares.pymain()10 Mins
    scheduled_tasks/discover/get_low_float.pymain()Daily (AH)
    scheduled_tasks/discover/get_high_interest.pymain()Daily (AH)
    scheduled_tasks/discover/get_earnings.pymain()Daily (AH)
    scheduled_tasks/discover/get_latest_insider_trading.pymain()2 Hours
    scheduled_tasks/discover/get_stocks_summary.pymain()10 Mins
    scheduled_tasks/discover/get_stocktwits_summary.pymain()Hourly
    scheduled_tasks/discover/get_ipo_calendar.pymain()Daily (AH)
    scheduled_tasks/discover/get_stock_splits.pymain()Daily (AH)
    scheduled_tasks/discover/get_dividends.pymain()Daily (AH)
    scheduled_tasks/discover/get_jim_cramer.pymain()Daily (AH)
    scheduled_tasks/government/get_senate_trading.pymain()Daily (AH)
    scheduled_tasks/government/get_house_trading.pymain()Daily (AH)
    scheduled_tasks/news/get_news.pymain()10 Mins
    scheduled_tasks/reset/reset_stocksera_trending.pyreset_trending_db()30 Mins