DOCSEARCHbyAlgolia
Submit Docs

powerplantmatching

repository·master·Indexed 42 days ago

https://github.com/pypsa/powerplantmatching

A toolset for cleaning, standardizing, and combining multiple power plant databases to create high-quality datasets for energy system modeling, specifically for PyPSA and TIMES. It provides modules for data ingestion, cleaning, matching, and collection, along with a specialized pandas accessor for power-plant-specific operations and visualization tools.

Tokens
19.7K
Snippets
83
Records
97
Agent score
48%

What's inside powerplantmatching

  1. Overview of powerplantmatching features

    master

    The powerplantmatching toolset is designed for cleaning, standardizing, and combining multiple power plant databases. It is particularly useful for creating high-quality inputs for PyPSA-based energy system models.

    Key capabilities include:

    • Data Processing: Cleaning and standardizing datasets, aggregating units belonging to the same plant, and comparing/combining different sources.
    • Capacity Management: Choosing between gross/net capacity and scaling capacities to match country-specific statistics.
    • Analysis & Visualization: Creating lookups, providing statistical insights into data quality, and visualizing power plant data.
    • Integration: Exporting processed powerplant data directly to PyPSA-based models.

  2. Overview of powerplantmatching

    master

    powerplantmatching is a toolset designed for cleaning, standardizing, and combining multiple power plant databases. It is primarily used to create high-quality power plant data inputs for energy system models like PyPSA or TIMES.

    Key capabilities include:

    • Data Processing: Cleaning, standardizing, and aggregating individual power plant units into single plant entities.
    • Data Integration: Comparing and merging multiple datasets (including a pre-merged dataset from six different sources).
    • Capacity Management: Choosing between gross and net capacity, and scaling capacities to match country-specific statistics.
    • Analysis & Visualization: Creating lookups, providing statistical insights into data quality, and visualizing power plant distributions.
    • Model Export: Exporting processed data directly to PyPSA or TIMES model formats.

  3. Overview of powerplantmatching data processing modules

    master

    The powerplantmatching library organizes its data processing capabilities into four primary modules:

    • powerplantmatching.data: Handles raw data ingestion and initial data structures.
    • powerplantmatching.cleaning: Provides tools for cleaning, filtering, and normalizing power plant datasets.
    • powerplantmatching.matching: Contains the core logic for matching power plant data against reference datasets.
    • powerplantmatching.collection: Manages the aggregation and collection of processed data.

  4. Understand the powerplantmatching dataset structure

    master

    The powerplantmatching dataset aggregates data from multiple sources into a unified format. Each entry in the dataset provides the following attributes:

    • Power plant name: The name claimed by the source database.
    • Fueltype: One of {Solid Biomass, Biogas, Geothermal, Hard Coal, Hydro, Lignite, Nuclear, Natural Gas, Oil, Solar, Wind, Other}.
    • Technology: One of {CCGT, OCGT, Steam Turbine, Combustion Engine, Run-Of-River, Pumped Storage, Reservoir}.
    • Set: Categorization into {Power Plant (PP), Combined Heat and Power (CHP), Storages (Stores)}.
    • Capacity: Measured in [MW].
    • Duration: Maximum state of charge capacity in hours at full output capacity.
    • Dam Information: Includes Dam volume [Mm^3] and Dam Height [m].
    • Geo-position: Latitude and Longitude.
    • Country: Covers EU-27 + CH + NO (+ UK), excluding Cyprus and Malta.
    • YearCommissioned: The year the powerplant was commissioned.
    • RetroFit: The year of the last retrofit.
    • projectID: An immutable identifier for the power plant.

  6. Set up a custom global configuration

    master

    You can customize the behavior of powerplantmatching by creating a global configuration file. This allows you to control the set of countries and fuel types, manage data source combinations, and apply filters to specific data sources.

    To set this up, save the default configuration file to ~/.powerplantmatching_config.yaml.

    Customization capabilities:

    • Define the global set of countries and fueltypes.
    • Determine which data sources to combine and which should be fully contained in the final dataset.
    • Filter individual data sources using pandas.DataFrame.query statements provided as arguments for the data source name.
    • Add an ENTSOE security token to enable manual updates of ENTSOE data.
    • Add a Google API key to enable geoparsing functionality.
    # Download the default config to your home directory (Linux/macOS)
wget -O ~/.powerplantmatching_config.yaml https://raw.githubusercontent.com/PyPSA/powerplantmatching/master/powerplantmatching/package_data/config.yaml

  7. Cite powerplantmatching in academic work

    master

    If you use powerplantmatching in your research or publications, please cite the following paper:

    F. Gotzens, H. Heinrichs, J. Hörsch, and F. Hofmann, "Performing energy modelling exercises in a transparent way - The issue of data quality in power plant databases", Energy Strategy Reviews, vol. 23, pp. 1–12, Jan. 2019, doi:10.1016/j.esr.2018.11.004.

  8. Download the pre-merged powerplant data

    master

    If you do not want to process the data yourself, you can directly download the current version of the cleaned and merged dataset as a CSV file from the repository's source.

    https://github.com/PyPSA/powerplantmatching/blob/master/powerplants.csv

  9. Report bugs and request features

    master

    To report bugs or request new features for powerplantmatching, please use the official GitHub issue tracker. When reporting issues, it is highly recommended to include a minimal working example (MWE) that illustrates the problem and a description of the approaches you have already attempted to resolve it.

    https://github.com/PyPSA/powerplantmatching/issues

  11. Install powerplantmatching via pip or conda

    master

    You can install the powerplantmatching package using either pip (or uv pip) or via conda/mamba from the conda-forge channel. The package is available on PyPI and Conda-Forge.

    # Using pip
pip install powerplantmatching

# Using conda or mamba
conda install -c conda-forge powerplantmatching

  12. Load power plant data with pm.powerplants()

    master

    Use pm.powerplants() to load power plant datasets into a pandas DataFrame.

    • To quickly load the pre-built dataset from a URL, set from_url=True. This is the fastest way to get started.
    • By default (from_url=False), the function loads and combines all necessary local data files. This process can take several minutes.
    • To get the full dataset including original data entries and links between matched plants and their source properties, set reduced=False.
    import powerplantmatching as pm

# Load the pre-built dataset directly from a URL
df = pm.powerplants(from_url=True)

# Load the full dataset with original entries (requires Java)
df_full = pm.powerplants(reduced=False)