DOCSEARCHbyAlgolia
Submit Docs

PyPSA-Eur

repository·master·Indexed 20 days ago

https://github.com/pypsa/pypsa-eur

An open-source, sector-coupled optimisation model and Snakemake-based workflow for modelling the European energy system. It provides a comprehensive dataset covering power, gas, hydrogen, and CO2 across multiple sectors including transport, heating, biomass, industry, and agriculture, designed as high-quality input for the PyPSA modelling framework.

Tokens
34K
Snippets
71
Records
163
Agent score
70%

What's inside pypsa-eur

  1. Overview of PyPSA-Eur capabilities

    master

    PyPSA-Eur is an open model dataset of the European energy system at the transmission network level, covering the full ENTSO-E area. It provides data for both electricity-only and sector-coupled energy systems.

    Electricity System

    Includes:

    • Alternating current (AC) lines at and above 220 kV.
    • High voltage direct current (HVDC) lines.
    • Substations.
    • An open database of conventional power plants.
    • Time series for electrical demand and variable renewable generator availability.
    • Geographic potentials for wind and solar power expansion.

    Sector-Coupled Energy System

    Includes all electricity features plus demand and supply for:

    • Transport.
    • Space and water heating.
    • Biomass.
    • Energy consumption in agriculture, industry, and industrial feedstocks.
    • Carbon management (capture, usage, and sequestration).

    Note: Since version v0.8.0, sector-coupling features are integrated directly into PyPSA-Eur, and the separate PyPSA-Eur-Sec is deprecated.

  2. Overview of PyPSA-Eur

    master

    PyPSA-Eur is an open-source, sector-coupled optimisation model of the European energy system at the transmission network level. It covers the full ENTSO-E area and includes multiple energy sectors: transport, heating, biomass, industry, and agriculture.

    Key components of the dataset include:

    • A power grid model (based on OpenStreetMap) for voltage levels above 220kV (or optionally above 60kV).
    • The powerplantmatching open power plant database.
    • Electrical demand time series from the ENTSO-E Transparency Platform.
    • Renewable time series and geographical potentials (wind/solar) generated using atlite.
    • Energy balances from Eurostat and JRC-IDEES.

    The model is designed to be imported into the PyPSA energy system modelling framework.

  3. Navigate the PyPSA-Eur folder structure

    master

    The repository is organized into the following directories:

    • scripts: Python scripts executed by Snakemake rules.
    • rules: Snakemake rules loaded in the Snakefile.
    • envs: Conda environment specifications (though using pixi is recommended).
    • data: Input data not produced by Snakemake rules.
    • cutouts: Raw weather data cutouts from atlite.
    • resources: Intermediate workflow results used by subsequent rules.
    • results: Solved PyPSA network data, summary files, and plots.
    • logs: Log files.
    • benchmarks: Snakemake benchmarks.
    • doc: Project documentation.
    • docker: Optional Docker environments.

  4. Understand the PyPSA-Eur licensing model

    master

    PyPSA-Eur uses a multi-license approach depending on the type of asset. When using the project, ensure you comply with the specific license associated with the component you are using:

    • Source Code: Licensed under MIT.
    • Documentation: Licensed under CC-BY-4.0.
    • Configuration Files: Mostly licensed under CC0-1.0.
    • Data Files (in repository): Licensed under CC-BY-4.0.

    Note on External Data: Input data that is not part of the repository is subject to different licenses and terms of use. Refer to the data_sources.md guide for details on external data usage.

  5. Understand the PyPSA-Eur network building process

    master

    The PyPSA-Eur energy system model is prepared using a series of snakemake rules. The process follows a specific pipeline:

    1. Data Retrieval: Since not all data is in the git repository, you must first use retrieve* rules to download necessary data bundles.
    2. Base Network Construction:
      • build_shapes generates GeoJSON files for countries, exclusive economic zones, and NUTS3 areas.
      • base_network builds the base network containing all buses, HVAC lines, and HVDC links, and calculates Voronoi cells for substations.
    3. Network Simplification: To make co-optimization computationally viable, the network is simplified via:
      • simplify_network: Transforms the grid into a 380 kV only equivalent network.
      • cluster_network: Uses k-means clustering to partition the network into a specified number of zones, reducing the network to one bus per zone.
    4. Resource & Profile Calculation:
      • build_powerplants: Allocates thermal power plant capacities using powerplantmatching to clustered regions.
      • determine_availability_matrix: Performs land eligibility analysis for PV, onshore, and offshore wind.
      • build_renewable_profiles: Calculates hourly capacity factors and installation potentials constrained by land-use within Voronoi cells.
      • build_hydro_profile: Calculates hourly per-unit hydro power availability.
    5. Final Assembly: The add_electricity and prepare_network rules combine all inputs into a final PyPSA network file, typically stored at networks/base_s_{clusters}_elec.nc.

  6. Overview of Industry Demand Modelling

    master

    The PyPSA-Eur model estimates industry energy demands and process emissions by combining material output data (from JRC-IDEES and other sources) with specific transformation ratios.

    Key Modelling Assumptions:

    • Emissions: Classified into energy-related (curbed by low-emission sources) and process-related (mitigated via alternative manufacturing or recycling).
    • Heat Supply:
      • Processes < 500°C: Supplied by solid biomass.
      • Processes > 500°C: Supplied by electrification or synthetic methane (hydrogen-based heat is currently not included).
      • Geothermal and solar thermal are disregarded for high-temperature process heat.
    • Electrification: Processes with fossil and electrified alternatives (e.g., glass manufacture) are assumed to be completely electrified. Current electricity demands (lighting, motors, etc.) remain electric.
    • Spatial Distribution: Industrial demand is distributed geographically using the Hotmaps Industrial Database, which maps energy-intensive sites across the EU28.

  7. Understand sector-specific spatial resolution models

    master

    PyPSA-Eur-Sec applies different spatial modeling approaches depending on the sector. While the electricity network is primarily nodal, other sectors use different abstractions:

    | Sector | Spatial Modeling Approach | | :--- | : | | Electricity network | Nodal | | Residential/Commercial Electricity demand | Nodal (distributed by population and GDP) | | Residential/Commercial Heating demand | Nodal (distributed by population) | | Industrial Electricity demand | Nodal (based on HotMaps database locations) | | Industrial Heat/Chemicals demand | Nodal (based on HotMaps database locations) | | Hydrogen network | Nodal (if activated in config) | | Methane network | Single node for Europe OR Nodal (if activated in config) | | Solid biomass | Single node for Europe OR Nodal (if activated in config) | | CO2 | Single node for Europe OR Nodal with pipelines (if activated in config) | | Carbonaceous fuels | Single node for Europe by default (can be regionally resolved) |

    Note on CO2: In single-node mode, transport and storage costs for sequestered CO2 are added. These costs can be adjusted in the configuration file.

  8. Understand the PyPSA-Eur Base Network dataset

    master

    The base network dataset provides a topologically connected representation of the European high-voltage transmission grid (220 kV to 750 kV). It is constructed from OpenStreetMap data and is intended for use in sector-coupled energy system optimization models.

    Key Characteristics:

    • Scope: Covers high-voltage lines and links across most European countries (e.g., DE, FR, GB, IT, ES, etc.).
    • Topology: Uses a heuristic cleaning process to handle incomplete or ambiguous electrical parameters.
    • Aggregation: Close substations within a 500 m radius are aggregated into single buses, though exact locations of underlying substations are preserved.
    • Identifiers: Unique identifiers for lines and links are maintained.
    • Source: OpenStreetMap via the Overpass turbo API.
    • License: ODbL.

  9. Understand the PyPSA-Eur validation methodology

    master

    The PyPSA-Eur model workflow is validated by comparing network optimization outcomes against historical European power system behavior.

    Key aspects of the validation setup include:

    • Data Source: Comparisons utilize data from the 2019 ENTSO-E Transparency Platform.
    • Price Modeling: The setup uses monthly varying fuel prices for gas, lignite, coal, and oil, as well as CO2 prices. These prices are generated using the build_monthly_prices script.

  10. Configure biogas transport and usage

    master

    The availability of biogas depends on the gas_network configuration:

    • If gas_network is false: Biogas is aggregated into a common European resource.
    • If gas_network is true: Biogas potential is associated with each specific node of origin.

    Note: The model requires biogas to be upgraded to natural gas quality (biomethane) before it can be fed into the general gas network.

    # gas_network: true/false

  11. Understand PyPSA-Eur Directory Structure and Resource Management

    master

    PyPSA-Eur uses specific configuration settings to organize model outputs and intermediate files:

    • run.name: Determines the subdirectory within the results folder (e.g., results/test-elec/networks/...). Final model outputs are always stored here.
    • run.shared_resources.policy: Determines how intermediate files are stored in the resources folder:
      • If policy is a string (e.g., "test"): Intermediate files are stored in resources/[policy]/... (e.g., resources/test/networks/...).
      • If policy is true: Files are shared between runs in resources/....
      • If policy is false: Files are not shared and are stored in resources/[run.name].
      • If policy is "base": Some common files are shared while others remain run-specific.

  12. How PyPSA-Eur configuration validation works

    master

    PyPSA-Eur uses Pydantic models to provide type checking, default values, and documentation for the Snakemake configuration.

    The validation system is composed of:

    • Pydantic models located in scripts/lib/validation/config/ which define all available options.
    • Auto-generated files: config/config.default.yaml (the default configuration used by Snakemake) and config/schema.default.json (the JSON schema).

    When you modify the Pydantic models, you must regenerate the YAML and JSON files to ensure the configuration remains in sync with the validation logic.