repository·main·Indexed 20 days ago
https://github.com/xdslproject/xdslA Python-native framework for building compiler infrastructure using SSA-based intermediate representations (IRs). xDSL enables prototyping compilers in Python with compatibility with the MLIR/LLVM ecosystem, featuring tools for IR manipulation, dialect definition, and a Python front-end via PyASTContext for mapping Python to MLIR.
In xDSL, canonicalization refers to a class of optimizations that transform Intermediate Representation (IR) to reduce complexity. These transformations make the IR easier to analyze for subsequent optimization passes.
Common examples include:
x + 0 -> x2 + 2 -> 4x * 1 -> xFor a deeper theoretical understanding, refer to the MLIR documentation on Canonicalization.
The pyast front-end is an ongoing work with several current constraints:
Assign/AnnAssign) or complex structures like classes.code_generation.py module walks the AST to generate xDSL, but it is currently limited in its scope.python_code_check.py:To begin using xDSL, you can follow these resources:
Certain xDSL subprojects require additional dependencies. These must be specified explicitly using extras. Examples include:
xdsl[gui]xdsl[jax]xdsl[riscv]pip install xdsl[gui]xDSL can use MLIR as a backend to generate executables, but this functionality requires a local installation of the LLVM project and MLIR.
mlir-opt, mlir-translate, and clang are available in your system's PATH by adding the LLVM build directory to it.export PATH=<insert-your-path>/llvm-project/build/bin:$PATHTo add a new benchmark, follow these three steps:
Create a function or method named time_*. You have two patterns:
Pattern A: Class-based (using BenchmarkClass)
Use this if you need a shared setup method. Inherit from BenchmarkClass from benchmarks.bench_utils.
Pattern B: Standalone function
Use this for simple, module-scope functions. These do not inherit from BenchmarkClass.
Note on @safe_to_repeat: If the benchmark call is independent and does not alter state that would affect subsequent runs (e.g., it only reads immutable inputs), decorate it with @safe_to_repeat from benchmarks.bench_utils.
If your benchmark requires specific xDSL operations or workloads, add them to benchmarks/workloads.py as a new class method on WorkloadBuilder.
Associate the benchmark with a name in the profile call at the bottom of your file using BenchmarkFunction from bench_utils.
import importlib
import xdsl.dialects.newdialect
from benchmarks.bench_utils import BenchmarkClass, safe_to_repeat
class ImportDialects(BenchmarkClass):
@safe_to_repeat
def time_newdialect_load(self) -> None:
"""Time loading the `newdialect` dialect.""
importlib.reload(xdsl.dialects.newdialect)
# To register for local profiling:
from bench_utils import BenchmarkFunction, profile
profile(
{
"Dialects.newdialect_load": BenchmarkFunction(time_newdialect_load),
}
)To use xDSL as part of a larger project for developing your own compiler, install the core package using pip.
pip install xdslBefore contributing or developing with xDSL, ensure you have performed a Developer Installation and configured Formatting and Typechecking as described in the main repository README.
To verify your environment is correctly configured, run the following command in your terminal. All tests must pass before proceeding with development tasks.
make testsxDSL provides introductory notebooks for compiler development using Marimo, a modern Python notebook that runs in your browser. Marimo maintains a dependency graph between cells, automatically re-running dependencies when a cell is changed.
Note: When you open a notebook, it automatically installs xdsl in a browser-based environment. You may need to wait for the installation to complete and then re-run all cells to begin working.
The xDSL interactive application is a command-line tool used to explore and construct compiler pipelines visually. It allows you to input xDSL IR, select passes from a tree, and see the resulting IR and operation count changes in real-time.
To run the GUI normally, execute the following command in your terminal:
xdsl-guiIf you are developing or need to run in development mode, ensure you have textual-dev installed, then run:
textual run xdsl.interactive.app:InputApp --devxdsl-guiBenchmarks are invoked by running the specific benchmark script (e.g., benchmarks/lexer.py) using uv run. The CLI requires two positional arguments: the name of the benchmark (or all) and the mode of execution.
Available Modes:
run: Execute the benchmark.timeit: Measure the execution time.snakeviz, viztracer, flameprof, pyinstrument: Profiling tools.dis: Disassemble the benchmark.Options:
-o, --output OUTPUT: Directory to write profile files.-q, --quiet: Suppress the profiler's UI.uv run benchmarks/lexer.py Lexer.empty_program runTo embed MLIR into Python using the xDSL front-end, you must first instantiate a PyASTContext. This context acts as a registry that maps Python types, methods, and literals to their corresponding xDSL IR types, operations, and attributes.
You must register the following for every element used in your program:
register_type(python_type, xdsl_type).register_function(python_callable, xdsl_op) to map Python operations (like __add__) to xDSL operations.register_literal(python_type, factory_func) where the factory function returns an xDSL operation representing the constant (e.g., a ConstantOp).from xdsl.dialects.arith import AddfOp, ConstantOp, MulfOp
from xdsl.dialects.builtin import FloatAttr, f64
from xdsl.frontend.pyast.context import PyASTContext
ctx = PyASTContext()
ctx.register_type(float, f64)
ctx.register_function(float.__add__, AddfOp)
ctx.register_function(float.__mul__, MulfOp)
ctx.register_literal(float, lambda v: ConstantOp(FloatAttr(v, 64)))