CompilerGym Documentation

CompilerGym is a toolkit designed to expose compiler optimization problems for reinforcement learning (RL). It serves two primary purposes:

1. For AI Researchers: It provides a high-quality, open-source OpenAI Gym environment for experimenting with program optimization techniques without requiring deep expertise in compiler internals.
2. For Compiler Developers: It provides a framework to expose new optimization problems to the AI community.

The toolkit aims to lower the barrier to entry for compiler AI research by providing a common experimental framework that improves fairness and reproducibility.

The Deep Q-Network (DQN) approach is a reinforcement learning method designed to learn sequences of transformation passes on programs. It uses a neural network to approximate Q-value iteration, where an agent interacts with the environment and stores transitions (state, action, reward, new state, done) in a replay buffer to remove sequential correlations.

Key characteristics:

• Goal: Learn optimal transformation sequences to optimize metrics like instruction count.
• Policy: The policy is deterministic after training, though initial network parameterization is non-deterministic.
• Learning Type: Off-policy learning using a replay buffer and a target network to stabilize updates via Huber loss.
• CompilerGym Version: 0.1.9

cBench is a suite of benchmarks used for evaluating compiler performance. You can find the project homepage and download the source files via the links below.

• Homepage: http://ctuning.org/wiki/index.php/CTools:CBench
• Download (V1.1): http://downloads.sourceforge.net/project/cbenchmark/cBench/V1.1/cBench_V1.1.tar.gz
• License: GNU General Public License Version 2

Tabular Q is a tabular, online Q-learning algorithm designed for CompilerGym. It is trained on each program in the test set individually. The algorithm computes the expected accumulated reward for state-action pairs and stores them in a table, updating these estimates immediately after each step taken in the environment (online learning).

Key Characteristics:

• Deterministic Policy: Once trained, the policy is deterministic.
• Non-deterministic Training: The training process itself is non-deterministic, meaning results may vary between training runs.
• Open Source: MIT licensed.

Many modifications to environments can be achieved by using or extending compiler_gym.wrappers.

Existing wrappers include:

• TimeLimit: Limits the length of episodes.
• ConstrainedCommandline: Constrains available actions.
• RandomOrderBenchmarks: Randomizes the benchmark selected on reset().

To implement custom logic like reward shaping, extend the base wrapper classes and implement the reward() method in a RewardWrapper subclass.

The CompilerEnvState class and its associated reader/writer components are used to manage the state of the compiler environment.

• CompilerEnvState: The core class representing the environment state.
• CompilerEnvStateWriter: Used to write or modify environment states.
• CompilerEnvStateReader: Used to read or iterate through environment states via the __iter__ method.

The compiler_gym.util.flags modules provide reusable command line flag definitions for compiler_gym.bin and other scripts. This prevents multiple-definition errors when scripts are imported.

Important Requirement: To use these flags, you must initialize the absl flags library. Because of this dependency, these flags are intended for use in scripts and CLI tools, not within the core library.

CompilerGym provides the LlvmEnv class (found in compiler_gym.envs.LlvmEnv) which exposes the LLVM 10.0.0 Intermediate Representation (IR) optimizer as a reinforcement learning environment. This allows agents to interact with LLVM's machine-independent IR.

from compiler_gym.envs import LlvmEnv

# Initialize the LLVM environment
env = LlvmEnv()

The CompilerEnv class provides access to available observation spaces and reward spaces through view objects. These views allow you to query lazily computed values at any point during the environment's lifetime.

• Use the env.observation attribute to access the ObservationView.
• Use the env.reward attribute to access the RewardView.

CompilerGym provides two ways to access the LLVM-IR representation of a module within the LLVM observation space:

1. Ir: Returns a serialized string representation of the LLVM-IR. This is useful for text-based processing or direct inspection.
2. BitcodeFile: Returns a string containing the file path to a serialized bitcode file (.bc) stored on disk. This is useful for passing the module to external tools that require bitcode files.

Note: Files generated via BitcodeFile are stored in a temporary directory and are automatically deleted when env.close() is called.

# Accessing the IR as a string
ir_string = env.observation["Ir"]

# Accessing the bitcode file path
bitcode_path = env.observation["BitcodeFile"]