DOCSEARCHbyAlgolia
Submit Docs

Sinergym

repository·main·Indexed 42 days ago

https://github.com/ugr-sail/sinergym

A Gymnasium-based interface for interacting with building simulation engines like EnergyPlus, specifically designed for reinforcement learning research in building energy optimization. Version 3.12.2 provides a framework to control simulations in real-time through custom controllers and RL agents, abstracting the EnergyPlus Python API to manage variables, meters, and actuators.

Tokens
67.1K
Snippets
184
Records
275
Agent score
53%

What's inside Sinergym

  1. What is Sinergym?

    main

    Sinergym is a Gymnasium-based interface designed to interact with building simulation engines, specifically the EnergyPlus Python API. It allows developers to control building simulations through time using custom controllers, making it an ideal framework for training and testing reinforcement learning (RL) agents in building energy management scenarios.

    Key capabilities include:

    • Simulation Engine Compatibility: Currently supports EnergyPlus via its Python API, with plans for engines like OpenModelica.
    • Benchmark Environments: Provides standardized environments to test and compare RL algorithms, similar to Atari or Mujoco.
    • Custom Experimentation: Allows users to customize reward functions, wrappers, and controllers, or create entirely new environments.
    • Actuator Control: Automates building actuator management through the Gymnasium interface; users only need to specify actuator names.
    • Integration: Highly integrated with Stable Baselines 3 (algorithms, wrappers, and callbacks) and Weights and Biases for automated logging and cloud storage.

  2. Use sinergym.utils.wrappers for environment modification

    main

    The sinergym.utils.wrappers module provides a collection of Gymnasium-style wrappers and utility functions designed to modify Sinergym environments. These wrappers allow you to transform observations, actions, rewards, and metadata to suit specific Reinforcement Learning (RL) tasks.

    Common Wrapper Categories:

    • Observation & Action Transformation: Wrappers like NormalizeAction, NormalizeObservation, ReduceObservationWrapper, and DiscretizeEnv modify the input/output space of the environment.
    • Reward Shaping: EnergyCostWrapper and MultiObjectiveReward allow for modifying the reward signal based on energy consumption or multiple objectives.
    • Context & State Management: PreviousObservationWrapper, ProbabilisticContextWrapper, ScheduledContextWrapper, and WeatherForecastingWrapper inject temporal or probabilistic context into the agent's view.
    • Logging & Monitoring: LoggerWrapper, CSVLogger, WandBLogger, and BaseLoggerWrapper provide mechanisms to track environment metrics and training progress.
    • Constraint & Smoothing: OfficeGridStorageSmoothingActionConstraintsWrapper and DeltaTempWrapper help manage physical or operational constraints within the simulation.

  3. Explore sinergym.utils modules

    main
    The sinergym.utils package provides a collection of utility modules for managing callbacks, common operations, constants, controllers, environment checking, Google Cloud integration, logging, reward calculation, serialization, and environment wrappers. Use these modules to extend or customize your Sinergym environment interactions.

  4. Use sinergym.utils.logger for logging

    main

    The sinergym.utils.logger module provides various logging utilities and handlers to manage output during Sinergym simulations. It includes specialized handlers for terminal output, progress bars (TqdmLoggingHandler), and integration with experiment tracking tools like Weights & Biases (WandBOutputFormat).

    Available classes for customizing logging behavior include:

    • CustomFormatter: For defining specific log message formats.
    • LoggerStorage: For managing log storage.
    • SimpleLogger: A basic logging implementation.
    • TerminalLogger: For logging directly to the terminal.
    • TqdmLoggingHandler: For integrating logs with tqdm progress bars.
    • WandBOutputFormat: For formatting logs for Weights & Biases.

  5. Explore Sinergym environments in sinergym.envs

    main
    The sinergym.envs module serves as the primary entry point for accessing various building energy simulation environments. Currently, it includes the eplus_env module, which provides environments based on EnergyPlus simulations. You can use this module to instantiate specific energy environments for reinforcement learning tasks.

  6. Understand Sinergym CI/CD workflows

    main

    Sinergym uses GitHub Actions to automate several processes based on repository events:

    Pull Requests

    • Formatting: Checks code against black and ruff (import sorting). Non-compliance triggers warnings.
    • Type Checking: Uses pyright to enforce I/O types for functions and methods. Type errors will halt the workflow.
    • Documentation: Compiles Sphinx documentation and runs spelling checks.
    • Testing: Builds a remote container via Dockerfile and executes pytest.
    • Security: Scans for differences between source and base in workflows to prevent malicious code in forks.

    Pushes to Main (or Merged PRs)

    • Auto-formatting: A bot applies black and ruff changes directly to the main branch.
    • Docs Deployment: A bot updates the documentation build on GitHub Pages.
    • Docker Hub Update: Builds and pushes containers to sailugr/sinergym using the latest tag and a lite tag.
    • Coverage: Runs tests and uploads reports to CodeCov.

    New Releases

    • Published/Edited Releases: Triggers a Docker Hub build with the specific release tag version and updates the PyPi repository.
    • Pre-releases: Updates the PyPi version but does not update the Docker Hub container.

  8. Details of the OfficeGridStorageSmoothing building environment

    main

    The OfficeGridStorageSmoothing.epJSON environment represents a large office building with 12 floors and a basement (total area: 46,320 m²).

    Structure

    The building is organized into 19 zones across several levels:

    • Basement
    • Bottom level (bot)
    • Mid level (mid)
    • Top level (top)

    Each level (bot, mid, top) consists of one core zone and four perimeter zones. Additionally, floor zones are described for the bot, mid, and top levels. This environment includes battery control capabilities for charging and discharging from the grid.

  9. Understand Sinergym building abstractions and customization

    main

    In Sinergym, buildings are distinct components integrated into environments. Each building comes with predefined default observation and action spaces.

    Key details for users:

    • Customization: While buildings have default observation and action spaces, these can be customized to suit specific research needs.
    • Observation Naming Convention: Observation variable names follow a specific pattern: they consist of the base_name prefixed with its corresponding key.

    If you intend to extend the library by adding a new building, you should follow the specific guidelines in the Adding new buildings section of the documentation.

  10. Configure Sinergym using modeling modules

    main
    Sinergym configuration is organized into specialized modules, primarily under the sinergym.config.modeling namespace. These modules allow you to define the parameters for building and running building energy simulation environments. Users should explore the modeling sub-modules to find specific configuration schemas for different building types or simulation setups.

  11. Record interaction data with LoggerWrapper

    main
    The LoggerWrapper is used to capture and store information during the controller-environment interaction flow. When using this wrapper, the environment object will gain a new attribute called data_logger, which is an instance of LoggerStorage containing the recorded information. This wrapper is customizable for using custom metrics and episode summary metrics.

  12. Retrieve evaluation results from LoggerEvalCallback

    main

    If you use Stable Baselines 3 and wrap your environment with LoggerEvalCallback, an evaluation/ directory is created in the root output folder. This directory contains:

    • best_model.zip: The best model weights obtained during training.
    • mean.txt, var.txt, count.txt: Normalization statistics specifically for the moments of best model evaluation.
    • evaluation_summary.csv: A CSV summary of the evaluation results, with the structure defined by the LoggerEvalCallback class.