Metal.jl

Metal.jl provides three distinct interfaces for Metal programming on Apple Silicon, allowing you to choose the level of abstraction required for your task:

1. MtlArray type: The primary interface for most users. It allows for array-based programming using platform-agnostic patterns like broadcasting and standard array abstractions.
2. Native kernel programming: Used when performance bottlenecks or specific functionality requirements necessitate writing custom Metal kernels directly in Julia.
3. Metal API wrappers: Provides low-level access for direct interaction with the underlying Metal libraries.

For most workflows, you can rely solely on MtlArray and standard Julia array operations.

The res/wrap directory contains scripts used to automate the generation of Julia wrappers for the following Apple frameworks:

• Metal
• Metal Performance Shaders (MPS)
• Metal Performance Shaders Graph

Important Limitations & Notes:

• Objective-C methods are currently not supported by the generation process.
• Scripts must be executed from within the res/wrap directory.
• The generation process supports the latest Julia release compatible with Metal.jl.

The MtlArray type is the core abstraction for managing GPU memory and moving data between the CPU and GPU. You can use it to allocate uninitialized memory on the GPU, copy data, fill arrays with values, and reshape them. Memory management is automatic; setting an MtlArray variable to nothing will trigger the release of the associated GPU memory.

# Allocate uninitialized memory on the GPU
a = MtlArray{Int}(undef, 1024)

# Perform memory operations
b = copy(a)
fill!(b, 0)

# Automatic memory management
a = nothing

The MtlArray type is the primary abstraction for managing device memory. It allows you to perform data-parallel operations on the GPU without writing custom kernels by leveraging Julia's broadcasting syntax.

a = MtlArray([1])
# Perform element-wise addition on the GPU
a .+ 1

Metal arrays (MtlArray) use different storage modes to determine how the GPU and CPU access the underlying memory.

• Metal.PrivateStorage: The default mode. The resource is accessible only by the GPU. This is typically the most performant for GPU-only computations.
• Metal.SharedStorage: The resource is accessible by both the CPU and GPU. This is useful for data that needs to be read or written by the CPU after GPU processing.
• Metal.ManagedStorage: The resource is accessible by both, but the system manages synchronization between the CPU and GPU caches.

Metal.jl provides the MtlArray type to leverage GPU parallelism. MtlArray implements the AbstractArray interface, meaning most standard Julia array operations work out of the box. For high-performance element-wise operations, use broadcasting or map. If you encounter 'scalar iteration' (slow performance due to CPU-GPU synchronization), check the issue tracker or use the underlying Metal APIs via the relevant submodules.

using Metal
a = MtlArray{Float32}(undef, (1, 2))
a .= 5.0
map(sin, a)

Metal.jl uses a hierarchical model for parallel execution. When you launch a kernel, it runs across multiple pseudo-independent instances called threads. These threads are organized into a three-tier hierarchy:

1. Thread: The single execution unit of the kernel.
2. Threadgroup: A collection of threads that share a common block of memory and synchronization barriers.
3. Grid: A collection of threadgroups.

You control the size and number of these units using the @metal macro's threads and groups keyword arguments. These dimensions can be 1, 2, or 3-dimensional.

# Example: Launching 3 threadgroups of 10x10 threads (total 300 threads)
@metal threads=(10,10) groups=3 my_kernel(gpu_image_array)

The Metal compiler runs as a system service and logs to the macOS unified logging system. You can view these logs using the Console app (ensure Include Info Messages and Include Debug Messages are enabled) or the log command-line tool.

To identify a crash, look for a MTLCompilerService BEGIN message that is not followed by a completion message. A successful compilation shows a BEGIN followed by a completion, whereas a crash results in a MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED error in the client process (e.g., Julia).

log show --last 2m --info --debug \
  --predicate 'process == "MTLCompilerService" OR (process BEGINSWITH "julia" AND eventMessage CONTAINS "Compil")'

To install Metal.jl, use the Julia package manager Pkg. This package provides the main entry point for GPU programming on MacOS in Julia, supporting various abstraction levels from high-level arrays to low-level Metal kernels.

using Pkg
Pkg.add("Metal")

If a kernel fails to compile, Metal.jl automatically writes the LLVM IR (.ll), AIR (.air), and Metal library (.metallib) to disk and prints the paths in the error message.

To capture the IR for every compiled kernel (not just failing ones), set the JULIA_METAL_DUMP_DIR environment variable to a directory of your choice. This is useful for debugging kernels that compile in Julia but fail on the hardware backend.

# Set the dump directory before running Julia
$ JULIA_METAL_DUMP_DIR=/tmp/metal-dumps julia

using Metal
# Every compiled kernel will now be dumped to /tmp/metal-dumps
@metal threads=length(c) vadd(a, b, c);