DOCSEARCHbyAlgolia
Submit Docs

Pirate Weather API Documentation

repository·main·Indexed 20 days ago

https://github.com/pirate-weather/pirateweather

A free, open weather forecast API designed as a drop-in replacement for the Dark Sky API, utilizing the same request syntax and JSON data structures. It provides global coverage for current conditions, minutely, hourly (up to 168 hours), and daily (7 days) forecasts, as well as historical data via the Time Machine API. The service supports various unit systems (SI, US, UK, CA) and integrates data from models such as GFS, HRRR, and ERA5.

Tokens
22.9K
Snippets
28
Records
101
Agent score
74%

What's inside Pirate Weather

  1. Overview of Pirate Weather API

    main

    Pirate Weather is a weather forecast service designed to provide weather data in a format compatible with the Dark Sky API style. It aims to make government-run weather model outputs (like NOAA/GFS) more accessible and web-friendly.

    Key features include:

    • Forecast data served in a Dark Sky-compatible format.
    • Historic data retrieval (similar to Dark Sky's Time Machine function).
    • Improved precipitation probability calculations (using ensemble members with accumulations > 1 mm).
    • A front-end viewer available at https://weather.pirateweather.net/.

    For detailed setup and usage instructions, visit the main website at https://pirateweather.net/.

  2. Usage limitations and service reliability for Pirate Weather

    main

    Pirate Weather acts as a translation layer for public gridded weather data. When using the API, be aware of the following:

    • Data Reliability: Data may be incorrect, contain errors, or be subject to unexpected downtime.
    • Critical Use Warning: The service is provided without warranty and should not be used for life or property critical applications.
    • Call Limits: Users are strictly prohibited from attempting to circumvent API call limits (e.g., by creating multiple accounts).
    • Service Model: The service is provided on an "AS IS" and "AS AVAILABLE" basis.

  3. Understand the Pirate Weather AWS Infrastructure architecture

    main

    Pirate Weather's infrastructure is designed for high-speed weather data retrieval using a multi-layered approach:

    • Ingestion & Processing: Python scripts orchestrated by AWS Step Functions run on AWS Fargate containers (using a custom ARM-based image). These scripts download raw data, process it, and chunk it into Zarr format.
    • Storage Strategy: Processed Zarr data is stored as .zip files on Amazon S3. To achieve low-latency IOPS, an rclone container syncs these files to autoscaled EC2 NVMe instances, serving data from local NVMe rather than S3 directly.
    • ECS Service: An ECS service manages four primary containers on EC2 instances: rclone (syncing), the production FastAPI container, a development container, a historic data (Time Machine) container, and Kong.
    • Traffic & Security: Inbound requests flow through Amazon CloudFront $\rightarrow$ Network Load Balancer (NLB) $\rightarrow$ EC2 instances $\rightarrow$ Kong Gateway. Kong handles authentication and rate limiting.
    • Data Persistence: Amazon ElastiCache (Redis) is used for session and rate-limit caching, while Amazon RDS stores persistent metadata and user information.

  4. Use the Pirate Weather API as a Dark Sky replacement

    main

    Pirate Weather is a free, open, and documented forecast API designed as a drop-in replacement for the Dark Sky API. It serves weather data in the same JSON structure used by Dark Sky, making it compatible with legacy applications like Home Assistant integrations, Magic Mirror cards, and other tools built for the Dark Sky ecosystem.

    Key features include:

    • Compatibility: Uses the same field names and JSON structure as Dark Sky.
    • Forecast Data: Provides current conditions, minutely data (1 hour), hourly data (up to 168 hours), and daily data (7 days).
    • Global Coverage: Accurate forecasts for anywhere in the world using GFS and HRRR models.
    • Weather Alerts: Supported for the US.
    • Time Machine: Access to historic weather data via the ERA5 dataset.

  5. Data retention and security policies

    main

    Pirate Weather retains personal information only as long as necessary to fulfill the purposes outlined in the privacy notice, or as required by law (e.g., tax or accounting requirements). Once there is no ongoing legitimate business need, data is either deleted or anonymized.

    Security is maintained through organizational and technical measures, but users should note that no internet transmission is 100% secure. Users are advised to access services within a secure environment.

  6. Understand what information Pirate Weather collects

    main

    Pirate Weather collects information in three main categories:

    1. Personal Information Provided by You: This includes names, email addresses, and passwords provided during registration or when contacting support.
    2. Payment Data: If you make purchases, payment instrument numbers and security codes are collected. Note: All payment data is stored by Stripe.
    3. Information Automatically Collected: When you use the services, the following are collected automatically:
      • Log and Usage Data: IP address, device information, browser type, settings, activity (date/time stamps, pages/files viewed, searches), and error reports.
      • Device Data: Computer/phone/tablet info, IP address, device/application IDs, location, browser type, hardware model, ISP, and OS configuration.
      • Location Data: Precise or imprecise geolocation data (e.g., via GPS or IP address).

  7. Forecast accuracy and data source limitations

    main

    When using the Pirate Weather API, be aware of the following data source behaviors and accuracy constraints:

    Minutely Forecasts

    • Within HRRR Domain: Minutely forecasts are calculated using HRRR-subhourly forecasts and are accurate to 15-minute periods.
    • Outside HRRR Domain: Minutely forecasts are calculated using GFS hourly forecasts, which provides significantly less granularity.

    Precipitation Probabilities

    Precipitation probabilities are derived from the Global Ensemble Forecast System (GEFS), a 30-member ensemble. The probability is calculated as the ratio of members predicting precipitation (e.g., if 1 member predicts precipitation, the probability is 1/30).

    Precipitation Type and Accumulation

    • GEFS data is used to predict precipitation type and accumulation.
    • A 1:10 snow-water ratio is assumed for accumulation calculations.

    Current Conditions

    • Current conditions are based on model results (specifically HRRR-subhourly) which assimilate observations, rather than using direct real-time observations.

  9. Understand the Pirate Weather API Response Structure

    main

    The API returns a JSON object containing weather data organized into several key blocks. The top-level properties include location metadata (latitude, longitude, timezone, offset, elevation) and several forecast blocks:

    • currently: Current weather conditions.
    • minutely: Minute-by-minute precipitation intensity for the next 60 minutes.
    • hourly: Hour-by-hour forecast. If extend=hourly is used, this provides 168 hours of data instead of the default 48.
    • daily: Day-by-day forecast for the next 7 days.
    • day_night: A 16-item list alternating between day (4:00 am to 4:59 pm) and night (5:00 pm to 3:59 am) forecasts for the next 7 days.
    • alerts: Severe weather alerts for the location.
    • flags: Metadata about the request (sources, units, version, etc.).
    {
  "latitude": 45.42,
  "longitude": -74.3,
  "timezone": "America/Toronto",
  "offset": -4.0,
  "currently": { ... },
  "hourly": { "data": [...] },
  "daily": { "data": [...] },
  "alerts": [ ... ],
  "flags": { ... }
}

  11. Understand the Pirate Weather API request structure

    main

    The Pirate Weather API is exposed via AWS API Gateway using the REST protocol. Requests are made to a custom domain name and rely on specific path and query parameters to function.

    Request Parameters

    • Path Parameters: The API extracts {api-key} and {location} directly from the URL path.
    • Query Parameters: The API also supports standard URL query parameters for additional configuration.

    Authentication Mechanism

    Unlike standard API implementations that require keys in headers, this service uses a custom Lambda Authorizer to allow the {api-key} to be passed as a path parameter. This is a compatibility measure that allows the API Gateway to validate the key even when it is part of the URL string. For this configuration to work, the API Key Source in the API Gateway settings must be set to AUTHORIZER.

  12. Understand the API Retrieval Logic

    main

    The retrieval Lambda function processes incoming requests using the following logic:

    • Input: A JSON payload containing latitude, longitude, extended forecast, and units.
    • Grid Mapping: Uses pickle files containing latitudes and longitudes to find the nearest grid cell.
    • Data Extraction: Iteratively extracts variables from NetCDF4 files into 2D numpy arrays.
    • Interpolation: Interpolates model data into minutely and hourly forecast arrays.
      • For true/false precipitation categories, interpolation uses 0 and 1, selecting the category with the highest value.
    • Spatial Averaging:
      • GFS/GEFS: Returns a weighted average (by 1/distance) of the closest 9 grid cells.
      • Mode: For non-averageable variables (like true/false), the most common (mode) result is used.
      • HRRR: Uses the closest cells directly due to high resolution.
    • Additional Calculations:
      • Timezone: Determined via timezonefinder.
      • Astral Data: Sunrise, sunset, and moon phases via astral.
      • Apparent Temperature: Calculated via wind chill or humidex.
      • UV Index: Calculated by multiplying modelled solar radiation by 0.4.
      • Alerts: Checks if the point is inside any GeoJSON polygons retrieved from the NWS API.