USAspending API

repository·master·Indexed 19 days ago

https://github.com/fedspendingtransparency/usaspending-api

A Django-based RESTful API that provides open-source federal spending data as part of the DATA Act. The system is supported by PostgreSQL and Elasticsearch and includes tools for managing bulk downloads, database migrations, and Elasticsearch index generation.

Tokens
187.7K
Snippets
530
Records
822
Agent score
66%

What's inside usaspending-api

  1. Understand the Disaster Spending response structure

    master

    A successful 200 OK response returns a JSON object containing three main components:

    1. totals: Aggregated spending data for the entire filtered set.

      • obligation (required, number)
      • outlay (required, number)
      • award_count (optional, number)
      • total_budgetary_resources (optional, number)
    2. results: An array of Result objects representing the hierarchy of Federal Accounts/TAS.

      • Each Result includes id, code, description, award_count, obligation, outlay, and total_budgetary_resources.
      • children (optional): An array of nested Result objects representing sub-accounts.
    3. page_metadata: Information about the current page and navigation.

      • page, next, previous, hasNext, hasPrevious, total, limit.
  2. How award filtering works in the USAspending API

    master

    Award filtering is implemented using the award.py and transaction.py modules. To filter results, you must pass a filters dictionary to the API. The structure of this dictionary (the available filter keys and their expected values) is defined in the project's external documentation schema.

    Currently, filtering is static, meaning the available filter keys are predefined in the source code.

    # Filtering is performed by passing a dictionary of filters
    filters = {
        "id": 1,
        "agency_name": "test"
    }
  3. Understand ETL Helper objects

    master

    The usaspending_api.common.etl.postgres module provides several abstraction objects to simplify database introspection and SQL generation during ETL (Extract, Transform, Load) processes. These objects reduce boilerplate when performing in-place creates, updates, and deletes on tables with many columns.

    Key objects include:

    • ETLTable: Represents a standard database table. It abstracts database introspection and encapsulates table properties.
    • ETLTemporaryTable: Functions like ETLTable but is specifically designed to work with temporary tables.
    • ETLDBLinkTable: A read-only ETLTable that resides on a remote server accessible via a dblink connection.
    • ETLQuery: A read-only object that treats a SQL query as if it were a table.
    • ETLQueryFile: A read-only object that treats a SQL query read from a file as if it were a table.
  4. Understand Disaster Spending Data Structures

    master

    The API uses specific objects to represent different financial aspects of disaster spending. Use the following logic to derive missing metrics from the provided fields:

    Funding (object)

    Represents the source funding for a specific code.

    • def_code (required, string): The DEF Code providing the source funding.
    • amount (required, number): The aggregation amount under that DEFC.

    Spending (object)

    Provides the core metrics for disaster spending.

    • award_obligations (required, number, nullable): Amount awarded and obligated.
    • award_outlays (required, number, nullable): Amount awarded, obligated, and outlayed.
    • total_obligations (required, number, nullable): Total amount obligated.
    • total_outlays (required, number, nullable): Total amount obligated and outlayed.

    Derived Metrics:

    • Other Obligations: total_obligations - award_obligations
    • Award Obligated But Not Yet Outlayed: award_obligations - award_outlays
    • Remaining Balance: total_budget_authority - total_obligations
    • Other Obligated But Not Yet Outlayed: (total_obligations - award_obligations) - (total_outlays - award_outlays)

    Additional (object)

    Used for special cases where financial details were not labeled with the searched DEFC.

    • total_budget_authority (required, number)
    • spending (object): Contains total_obligations and total_outlays.
  5. Combine Search Filters using AND/OR logic

    master

    When constructing a search request, you can combine multiple filter objects into a single JSON object. By default, different filter keys are combined using AND logic (e.g., a keyword filter AND a time period filter). However, some specific filters (like agencies or time_period arrays) allow for internal OR logic within their own structure.

    {
        "keywords": ["example search text"],
        "time_period": [
            {
                "start_date": "2001-01-01",
                "end_date": "2001-01-31"
            }
        ]
    }
  6. Understand the AwardingAgencyOfficeMatchObject response structure

    master

    The API returns a 200 OK response containing a results array of AwardingAgencyOfficeMatchObject items and a messages array for warnings or instructions.

    An AwardingAgencyOfficeMatchObject can represent one of three hierarchical levels:

    1. Top-tier Agency: Contains toptier_agency details, an array of subtier_agencies, and an array of offices.
    2. Sub-tier Agency: Contains subtier_agency details, the parent toptier_agency, and an array of offices.
    3. Office: Contains office details, the parent toptier_agency, and the parent subtier_agency.
  7. Understand the CFDA loan spending response structure

    master

    A successful 200 OK response returns a JSON object containing three main components:

    1. totals (required): Aggregated financial data for the entire filtered set, including award_count, face_value_of_loan, obligation, and outlay.
    2. results (required): An array of Result objects. Each object contains detailed program information such as code, description, cfda_federal_agency, cfda_objectives, and eligibility details (applicant_eligibility, beneficiary_eligibility).
    3. page_metadata (required): Information about the current page, including page, next, previous, hasNext, hasPrevious, total, and limit.
  8. Understand the response structure for disaster spending

    master

    A successful 200 OK response returns a JSON object containing three main components:

    1. totals (required): Aggregated spending data for the entire filtered set.

      • obligation (required, number)
      • outlay (required, number)
      • award_count (optional, number)
      • total_budgetary_resources (optional, number)
    2. results (required): An array of Result objects representing the agencies.

      • id (required, string)
      • code (required, string)
      • description (required, string)
      • award_count (required, nullable, number)
      • obligation (required, nullable, number)
      • outlay (required, nullable, number)
      • total_budgetary_resources (required, nullable, number)
      • children (optional, array of Result objects)
    3. page_metadata (required): Information about the current page and navigation.

      • page (required, number)
      • total (required, number)
      • limit (required, number)
      • next (required, nullable, number)
      • previous (required, nullable, number)
      • hasNext (required, boolean)
      • hasPrevious (required, boolean)
  9. API Contract Template (Format 1A)

    master

    This document defines the standard template (Format 1A) used for documenting USAspending API endpoints. It provides a structured way to describe endpoint paths, HTTP methods, required parameters, and the expected JSON response structures (including arrays and single objects).

    # Short Endpoint Name [/api/v2/this/is/your/{param_for_endpoint}/]
    
    ## GET
    
    + Parameters
        + `param_for_endpoint`: `endpoint` (required, string)
    
    + Response 200 (application/json)
        + Attributes
            + `data_structure_array` (required, array[ForTheArray], fixed-type)
            + `single_data_structure` (optional, SampleSingleObject)
            + `name` (required, enum[string])
    
    # Data Structures
    
    ## ForTheArray
    + `value_1` (optional, string)
    + `value_2` (optional, string)
    
    ## SampleSingleObject
    + `value_1` (required, number)
    + `value_2` (optional, string)
    + `value_3` (optional, number, nullable)