DOCSEARCHbyAlgolia
Submit Docs

CityLearn Documentation

repository·master·Indexed 20 days ago

https://github.com/citylearn-project/citylearn

An open-source Gymnasium environment for Multi-Agent Reinforcement Learning (RL) applied to building energy coordination and demand response in urban districts. It provides energy models for buildings and Distributed Energy Resources (DER), including heat pumps, electric heaters, and batteries. The library includes a comprehensive KPI v2 naming convention for analyzing building and district-level metrics, a visual dashboard (CityLearn UI) for simulation analysis, and support for the CityLearn Challenges.

Tokens
16.8K
Snippets
38
Records
83
Agent score
69%

What's inside CityLearn

  1. Overview of CityLearn

    master

    CityLearn is an open-source Farama Foundation Gymnasium environment designed for implementing Multi-Agent Reinforcement Learning (MARL) to manage building energy coordination and demand response in urban settings.

    Its primary goal is to help reshape the aggregated electrical demand curve of districts and cities. This is achieved by controlling:

    • Active energy storage for load shifting.
    • Heat pumps or electric heaters for load shedding.

    By flattening, smoothing, and reducing the overall electrical demand curve, CityLearn helps reduce the operational and capital costs of electricity generation, transmission, and distribution networks. The environment is standardized to facilitate the comparison of different RL algorithms in demand response tasks.

  2. Overview of CityLearn environment components

    master

    CityLearn provides energy models for buildings and Distributed Energy Resources (DER).

    Key Components:

    • Buildings: A collection of building energy models forms a virtual district (neighborhood/community).
    • End-use Loads: Includes space cooling, space heating, and domestic hot water.
    • Distributed Energy Resources (DER): Includes air-to-water heat pumps, electric heaters, and batteries.
    • Load Satisfaction: Space heating and domestic hot water can be satisfied via air-to-water heat pumps or electric heaters.

  3. Understand the CityLearn environment model

    master

    The CityLearn environment is a simulation of virtual districts composed of buildings with various energy models. It models the interaction between distributed energy resources (DERs), thermal and electrical loads, and the grid.

    Key Components:

    • Buildings: Collections of energy models that form a district. Each building contains end-use loads (space cooling, space heating, and domestic hot water).
    • Distributed Energy Resources (DERs): Includes air-to-water heat pumps, electric heaters, and batteries.
    • Energy Storage Systems (ESS): Thermal storage tanks and batteries used for load shifting and providing grid flexibility.
    • Generation: Photovoltaic (PV) arrays can be included to offset grid electricity consumption.
    • Loads: Includes shiftable loads (via storage) and non-shiftable (plug) loads.

    Control Logic: Agents (such as RBC, RL, or MPC) control the environment by determining the supply power for cooling/heating devices and the charge/discharge rates for active storage devices at each control time step.

  4. Understand the multi-objective cost function

    master

    The performance of an RL agent is measured by a multi-objective cost function composed of 5 equally weighted metrics. The goal is to minimize this cost. The cost is normalized against a baseline Rule-Based Controller (RBC); a score < 1 indicates the RL agent outperformed the RBC.

    The 5 metrics are:

    1. Peak demand: The maximum demand for the entire simulated period.
    2. Average daily peak demand: The average of the daily peak demands over a year.
    3. Ramping: The rate of change in demand.
    4. 1 - Load factor: A metric where a value closer to 0 indicates a load factor closer to 1.
    5. Net electricity consumption: The total electricity used.

  5. Simulate power outages

    master
    Since version 2.1.0, you can simulate power outages. During an outage, buildings cannot rely on the grid and must satisfy end-use loads using only available distributed energy resources (DERs), such as storage devices and PV systems. Failure to meet these loads results in thermal discomfort and unserved energy. During normal operation, the grid supply is considered unlimited.

  6. Distinguish between Solar Self-Consumption and Community Market Import Share

    master

    These two metrics measure different aspects of energy usage and should not be confused.

    1. Normal Solar Self-Consumption (*_solar_self_consumption_ratio_self_consumption_ratio)

      • Formula: (generation_total - export_total) / generation_total
      • Meaning: The fraction of PV generation consumed locally by the building/district. It is solar-centric and exists regardless of whether a community market is active.

    2. Community Market Import Share (district_solar_self_consumption_community_market_import_share_ratio)

      • Formula: community_local_traded_total / district_energy_grid_total_import_control
      • Meaning: The fraction of the district's total import demand from the grid that is covered by local community trading. It is market-centric.
      • Availability: Only available at the district level and only when community_market.enabled = true and community_market.kpis.community_self_consumption_enabled = true.

  7. Simulate dynamic indoor temperatures and load shedding

    master

    Since version 2.0.0, CityLearn supports dynamic indoor dry-bulb temperatures, allowing for load shedding energy flexibility services.

    Unlike legacy versions where temperatures were static and pre-computed, current versions use an LSTM model to encode a building's temperature evolution based on the supplied cooling or heating energy. This allows users to control the power of cooling and heating devices to influence temperature changes and achieve partial load satisfaction.

  8. Understand CityLearn control architectures

    master

    CityLearn supports three distinct control architectures for managing storage, cooling, and heating devices:

    1. Centralized: A single agent controls all devices across the entire district. The agent's action space is equal to the total number of storage, cooling, and heating devices in the district.
    2. Decentralized-independent: Each building has its own unique agent that acts in isolation. Agents do not share information. Each agent's action space is equal to the number of devices within its specific building.
    3. Decentralized-coordinated: Similar to the decentralized-independent architecture, but agents share information with one another to coordinate actions.

    Choosing an architecture depends on whether you want to model a single district-wide controller or multiple autonomous building-level controllers.

  9. Understand the structure of CityLearn observations

    master

    Observations in CityLearn are categorized into four groups: calendar, weather, district, and building.

    • Shared Observations: Values in the calendar, weather, and district categories are identical for all buildings within a single environment.
    • Building-Specific Observations: Values in the building category are unique to each individual building.

    Observations are handled in two ways:

    1. Static/Pre-calculated: Data can be supplied via .csv files (pre-calculated, pre-simulated, or pre-measured).
    2. Dynamic: Certain observations are calculated during simulation runtime based on the actions taken by agents.

  10. Understand CityLearn Datasets and Data Files

    master
    A CityLearn dataset is a collection of data files that define the simulation environment and provide agent-agnostic observations (values that do not change based on agent control actions). These files are referenced in a schema.json file and are read during the initialization of the citylearn.citylearn.CityLearnEnv class.

  12. Understand the meaning of actions in CityLearn

    master

    In CityLearn, an action is represented as a real number within the range [-1.0, 1.0]. This value prescribes the proportion of a storage device's total capacity that should be either charged or discharged.

    • Positive values (0.0 to 1.0): Represent the proportion of capacity to be charged.
    • Negative values (-1.0 to 0.0): Represent the proportion of capacity to be discharged.
    • Zero (0.0): Represents no change in storage state.