sumo-rl

The action space in SUMO-RL is discrete. At every delta_time interval, each traffic signal agent selects a discrete action representing the next green phase configuration.

Key Behavior:

• Phase Selection: An action corresponds to choosing a specific green phase configuration.
• Yellow Phase Transition: Whenever a phase change occurs (transitioning from one green phase to another), the system automatically inserts a yellow phase. This yellow phase lasts for a duration defined by yellow_time seconds.

To implement a custom observation function in sumo-rl, you must define a new class that inherits from sumo_rl.environment.observations.ObservationFunction. This allows you to define how the environment state is transformed into the observation space used by your agent.

from sumo_rl.environment.observations import ObservationFunction

class MyCustomObservation(ObservationFunction):
    def __init__(self, ...):
        super().__init__(...)
        # Initialize your custom observation logic
        pass

    def step(self, ...):
        # Implement the logic to return the observation
        pass

To use SUMO-RL, you must first install the SUMO simulator and set the SUMO_HOME environment variable.

1. Install SUMO (Linux/Ubuntu)

sudo add-apt-repository ppa:sumo/stable
sudo apt-get update
sudo apt-get install sumo sumo-tools sumo-doc

Set the SUMO_HOME variable (default path is /usr/share/sumo):

echo 'export SUMO_HOME="/usr/share/sumo"' >> ~/.bashrc
source ~/.bashrc

Performance Tip: For an ~8x performance boost using Libsumo, set:

export LIBSUMO_AS_TRACI=1

Note: Enabling this prevents running with sumo-gui or running multiple simulations in parallel.

2. Install SUMO-RL

Install the stable release via pip:

pip install sumo-rl

Or install the latest unreleased version from source:

git clone https://github.com/LucasAlegre/sumo-rl
cd sumo-rl
pip install -e .

You can change the reward calculation by passing a custom function to the reward_fn parameter in the SumoEnvironment constructor.

Available options include:

1. Pre-implemented functions: Various reward functions are available within the TrafficSignal class. Check the sumo_rl/environment/traffic_signal.py source for a list of implemented functions.
2. Custom functions: You can define your own function that accepts a traffic_signal object as an argument and returns a numeric reward value.

def my_reward_fn(traffic_signal):
    return traffic_signal.get_average_speed()

env = SumoEnvironment(..., reward_fn=my_reward_fn)

For multi-agent environments (multiple traffic lights), use the PettingZoo API via sumo_rl.parallel_env. This provides a parallel environment interface where actions are passed as a dictionary mapping agents to their chosen actions.

import sumo_rl

env = sumo_rl.parallel_env(net_file='nets/RESCO/grid4x4/grid4x4.net.xml',
                  route_file='nets/RESCO/grid4x4/grid4x4_1.rou.xml',
                  use_gui=True,
                  num_seconds=3600)

observations = env.reset()
while env.agents:
    # Sample actions for all agents
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, terminations, truncations, infos = env.step(actions)

The sumo_rl.environment.env.SumoEnvironment class is the primary interface for interacting with SUMO (Simulation of Urban MObility) via a Reinforcement Learning interface. It provides an environment that can be used with both single-agent (Gymnasium) and multi-agent (PettingZoo) APIs.

Note: Detailed member documentation is generated via Sphinx/autodoc. Users should refer to the specific class methods for managing simulation steps, resetting the environment, and retrieving observations, actions, and rewards.

from sumo_rl import SumoEnvironment

# Example initialization (exact parameters depend on specific SUMO configuration)
env = SumoEnvironment(
    # parameters go here
)

The sumo_rl.environment.observations.DefaultObservationFunction is the standard implementation provided by the library. It provides a baseline observation space for the SUMO environments.

from sumo_rl.environment.observations import DefaultObservationFunction

# Use the default observation function in your environment setup
obs_func = DefaultObservationFunction()

The sumo_rl.environment.env.SumoEnvironmentPZ class provides a multi-agent interface for SUMO simulations using the PettingZoo API. This allows you to interact with the SUMO environment as a multi-agent reinforcement learning problem, where multiple agents can observe the state and take actions simultaneously within the simulation.

from sumo_rl import SumoEnvironmentPZ

# Example initialization (exact parameters depend on specific environment configuration)
env = SumoEnvironmentPZ(...)

The sumo_rl.environment.traffic_signal.TrafficSignal class provides a Gymnasium-compatible environment for training reinforcement learning agents to control traffic signals in a SUMO simulation. It models a single intersection or a set of intersections where the agent's goal is to manage signal timings to optimize traffic flow.

from sumo_rl import TrafficSignal

# Example initialization (exact parameters depend on the specific SUMO network configuration)
env = TrafficSignal(
    net_file='path/to/your/network.net.xml',
    route_file='path/to/your/routes.rou.xml',
    # ... other configuration parameters
)