SMACv2 (StarCraft Multi-Agent Challenge v2)

SMAC (StarCraft Multi-Agent Challenge) is a research platform designed for evaluating Multi-Agent Reinforcement Learning (MARL) algorithms. Unlike standard StarCraft II gameplay which includes high-level strategy (macromanagement), SMAC focuses exclusively on micromanagement.

Key characteristics of the SMAC environment include:

• Decentralized Control: Each unit is controlled by an independent agent.
• Partial Observability: Agents condition only on local observations restricted to a limited field of view centered on that specific unit.
• Cooperative Combat: Groups of agents must learn to solve combat scenarios against a scripted AI opponent.
• Complex Behaviors: The environment is designed to test skills such as focus fire (jointly attacking specific targets), avoiding overkill, kiting (maintaining distance while attacking), and utilizing formations or terrain.

SMAC provides two types of information: local observations for agents and a global state for centralized training.

Local Observations

Agents receive partially observable information within a circular field of view (radius of 9). The observation includes:

• Unit attributes (for both allies and enemies in sight): distance, relative x, relative y, health, shield, and unit_type.
• Self-information: The agent's own health, shield, and unit_type are included.
• Action history: The last actions of allied units within the field of view.
• Terrain: Eight points at a fixed radius indicating height and walkability.

Global State

Used during centralized training, the state vector contains:

• Coordinates: All agents' coordinates relative to the map center.
• Unit features: All unit features present in observations.
• Special attributes: energy for Medivacs and cooldown for other allied units (minimum delay between attacks).
• Action history: The last actions of all agents.

All features are normalized by their maximum values.

SMAC (StarCraft Multi-Agent Challenge) consists of micro scenarios designed to evaluate cooperative Multi-Agent Reinforcement Learning (MARL). Scenarios involve a confrontation between two armies: the first army is controlled by learned allied agents, and the second consists of enemy units controlled by a built-in game AI using scripted heuristics.

Scenarios are categorized by their structure:

• Symmetric: Both armies have equal compositions.
  • Homogeneous: Each army has only one unit type (e.g., Marines).
  • Heterogeneous: Armies have multiple unit types (e.g., Stalkers and Zealots).
• Asymmetric: The enemy army outnumbers the allied army.
• Micro-trick: Scenarios requiring specific coordination techniques, such as:
  • Alternating fire: (e.g., 2m_vs_1z)
  • Kiting: (e.g., 3s_vs_3z)
  • Focus fire: (e.g., 6h_vs_8z)
  • Wall off: (e.g., corridor)
  • Positioning: (e.g., so_many_banelings)

The goal is to maximize the win rate (ratio of games won to games played).

To run the MAPPO baseline, follow these steps:

1. Ensure WANDB_API_KEY_FILE is set.
2. Clone the mappo repository: git clone https://github.com/benellis3/mappo.git.
3. Build the Docker container by running build.sh inside the docker directory.
4. Install Starcraft in the mappo directory: ./install_sc2.sh.
5. Configure src/config/default.yaml: set project to your desired name and entity to your WandB username.
6. Modify run.sh:
   • Set lr to 0.0005.
   • Set clip_range to 0.1.
   • Closed-loop baseline: Set maps to only contain maps without open_loop in their name.
   • Open-loop baseline: Set maps to only contain maps with open_loop in their name.
7. Execute the experiment: ./run.sh clipping_rnn_central_V <tag>

Note on Scenarios: For 10_vs_11 or 20_vs_23 scenarios, set the offset on line 21 of run.sh to control the enemy-to-ally ratio.

git clone https://github.com/benellis3/mappo.git
# ... inside mappo ...
./docker/build.sh
./install_sc2.sh
./run.sh clipping_rnn_central_V my_experiment_tag

Open-loop baselines require specific git branches.

MAPPO Open-Loop:

1. Clone mappo and ensure WANDB_API_KEY_FILE is set.
2. Checkout the stochastic-experiment branch: git checkout stochastic-experiment.
3. Install Starcraft: ./install_sc2.sh.
4. Build Docker: ./build.sh in the docker directory.
5. Configure src/config/default.yaml (project and entity).
6. Run: ./run_exp.sh clipping_rnn_central_V <tag>

QMIX Open-Loop:

1. Clone pymarl2 and ensure WANDB_API_KEY_FILE is set.
2. Checkout the stochastic_test branch: git checkout stochastic_test.
3. Install Starcraft: ./install_sc2.sh in the pymarl2 directory.
4. Build Docker: ./build.sh in the docker directory.
5. Configure src/config/default.yaml (project and entity).
6. Run: ./run.sh qmix <tag>

# MAPPO Open-Loop
git checkout stochastic-experiment
./run_exp.sh clipping_rnn_central_V <tag>

# QMIX Open-Loop
git checkout stochastic_test
./run.sh qmix <tag>

To run the QMIX baseline, follow these steps:

1. Ensure WANDB_API_KEY_FILE is set.
2. Clone the pymarl2 repository: git clone https://github.com/benellis3/pymarl2.git.
3. Build the Docker container from the pymarl2 directory: docker build -t pymarl2:ben_smac -f docker/Dockerfile --build-arg UID=$UID .
4. Install Starcraft in the pymarl2 directory: ./install_sc2.sh.
5. Configure src/config/default.yaml: set project to your desired name and entity to your WandB username.
6. Modify run_exp.sh:
   • Set td_lambdas (line 21) to 0.4.
   • Set eps_anneal to 100000.
7. Execute the experiment: ./run_exp.sh qmix <tag>

Note on Scenarios: For 10_vs_11 or 20_vs_23 scenarios, you must manually set n_units and n_enemies in the ./run_docker.sh command (line 46). For example, for 20_vs_23, use n_units=20 and n_enemies=23.

git clone https://github.com/benellis3/pymarl2.git
# ... inside pymarl2 ...
docker build -t pymarl2:ben_smac -f docker/Dockerfile --build-arg UID=$UID .
./install_sc2.sh
./run_exp.sh qmix my_experiment_tag