poke-env

The poke-env documentation provides several end-to-end examples tailored to different development goals. Depending on your objective, you should follow one of these specific guides:

• First local battle and custom agent: Follow the quickstart guide. This requires a local Pokemon Showdown server.
• Custom team selection and generation: Follow the using_a_custom_teambuilder guide. This requires a local Showdown server.
• Connecting to official or custom servers to challenge humans: Follow the connecting_to_showdown_and_challenging_humans guide. Note that an account is required for the official public server.
• Reinforcement Learning (RL) training: Follow the reinforcement_learning guide. This requires a local server and RL-specific dependencies (like Stable-Baselines3).
• Self-play training: Follow the self_play guide. This uses SuperSuit and Stable-Baselines3 and requires a local server.
• Custom RL pipelines (Action mapping and strict modes): Follow the action_mapping_and_strict_modes guide to understand environment actions, legality checks, and masks.
• Debugging battle state issues: Follow the strict_battle_tracking guide if you are experiencing request parsing or battle-state desynchronization issues.
• Replay persistence: Follow the saving_replays guide to learn about automatic replay persistence and explicit HTML replay export.

poke-env environments provide observations as dictionaries containing "observation" and "action_mask" keys. To prevent a reinforcement learning agent from selecting illegal moves, you must implement a custom policy in Stable-Baselines3 that applies this mask to the action logits.

1. Custom Features Extractor: Create a BaseFeaturesExtractor that pulls the "observation" tensor from the dictionary.
2. Masked Policy: Subclass ActorCriticPolicy (or similar) to intercept the "action_mask" and apply it as -inf to the action logits in _get_action_dist_from_latent. This ensures illegal actions have a probability of zero.

class MaskedActorCriticPolicy(ActorCriticPolicy):
    def __init__(self, *args, **kwargs):
        super().__init__(
            *args,
            **kwargs,
            net_arch=[64, 64],
            features_extractor_class=FeaturesExtractor,
        )

    def forward(self, obs, deterministic=False):
        self._mask = obs["action_mask"]
        return super().forward(obs, deterministic)

    def evaluate_actions(self, obs, actions):
        self._mask = obs["action_mask"]
        return super().evaluate_actions(obs, actions)

    def _get_action_dist_from_latent(self, latent_pi):
        action_logits = self.action_net(latent_pi)
        mask = torch.where(self._mask == 1, 0, float("-inf"))
        return self.action_dist.proba_distribution(action_logits + mask)

To create a custom agent, inherit from Player and override the choose_move method. This method receives a Battle object representing the current state and must return a valid battle order.

Key components for implementation:

• battle.available_moves: A list of Move objects available this turn.
• self.create_order(move_or_pokemon, ...): A helper to generate valid battle messages. You can pass a Move or a Pokemon object. For moves, you can specify parameters like terastallize=True.
• self.choose_random_move(battle): A fallback method that returns a valid random action (move or switch) if no specific move is chosen.

from poke_env.player import Player

class MaxDamagePlayer(Player):
    def choose_move(self, battle):
        if battle.available_moves:
            # Logic to pick the best move
            best_move = max(battle.available_moves, key=lambda move: move.base_power)
            
            # Example of using terastallize
            if battle.can_tera:
                return self.create_order(best_move, terastallize=True)
            return self.create_order(best_move)
        else:
            # Fallback to a random valid action
            return self.choose_random_move(battle)

The Player class provides a strict_battle_tracking option to aggressively validate internal battle-state consistency against Pokemon Showdown requests. This is primarily used for debugging parser or state issues rather than standard training runs.

When to use it

• Debugging battle-state desynchronization issues.
• Validating custom battle parsing logic or advanced agents.
• Developing against multiple formats and requiring early failure on inconsistencies.

Caveats

• Enabling this mode may raise assertions if Showdown requests and the locally tracked state disagree.
• In scenarios involving heavy use of illusions, special handling may be required in your test setup.
• For production runs where resilience is preferred over strict validation, keep strict_battle_tracking=False.

from poke_env.player import RandomPlayer

player = RandomPlayer(
    battle_format="gen9randombattle",
    strict_battle_tracking=True,
)