MetalPetal Documentation

MetalPetal is built around four primary abstractions that work together to perform image processing:

• MTIContext: The evaluation context used for rendering MTIImages. It manages caches and state information. For optimal performance, you should reuse a single MTIContext instance whenever possible rather than creating new ones for every operation.
• MTIImage: A representation of an image (or a recipe to produce one). It does not hold raw bitmap data directly; instead, it contains an MTIImagePromise (the recipe) and metadata like cachePolicy and samplerDescriptor. It is immutable and thread-safe.
• MTIFilter: Represents an image processing effect with controllable parameters. A filter takes input MTIImages and produces an output MTIImage. Note that MTIFilter objects are mutable and are not thread-safe.
• MTIKernel: The underlying image processing routine. It is responsible for creating the Metal pipeline states (render or compute) and building the MTIImagePromise for the resulting MTIImage.

MetalPetal provides a type-safe Swift API for building filter chains using the => operator within a FilterGraph.makeImage block.

• Unary Filters: MTIUnaryFilter instances can be connected directly using =>.
• Multi-input Filters: For filters with multiple inputs, you must connect to a specific inputPort (e.g., filter.inputPorts.inputImage).
• Constraints: The => operator is only valid inside FilterGraph.makeImage. Exactly one filter's output must be connected to the output parameter.

// Simple chain
let image = try? FilterGraph.makeImage { output in
    inputImage => saturationFilter => exposureFilter => output
}

// Complex chain with multiple inputs
let image = try? FilterGraph.makeImage { output in
    inputImage => saturationFilter => exposureFilter => contrastFilter => blendFilter.inputPorts.inputImage
    exposureFilter => blendFilter.inputPorts.inputBackgroundImage
    blendFilter => output
}

MTIContext

MTIContext is a heavyweight object. Reuse a MTIContext whenever possible. Create it early and use it for all subsequent rendering tasks.

MTIImage Cache Policy

Use MTIImage.cachePolicy to manage memory efficiency:

• MTIImageCachePolicyTransient: Use this for intermediate results in a filter chain. It allows the underlying texture to be reused, making it the most memory-efficient option. Note that requesting the output of a previously rendered image might trigger a re-render.
• MTIImageCachePolicyPersistent: Use this when you want to prevent the underlying texture from being reused (e.g., for images created from external sources).

MTIFilter.outputImage

MTIFilter.outputImage is a compute property. Accessing it may return a new object even if inputs haven't changed. To optimize, reuse output images when multiple filters share the same input:

// BETTER: Reuse the output image
let filterOutputImage = filterA.outputImage
filterB.inputImage = filterOutputImage
filterC.inputImage = filterOutputImage

// AVOID: This may create multiple redundant objects
filterB.inputImage = filterA.outputImage
filterC.inputImage = filterA.outputImage

For complex filters involving multiple inputs or specific logic, create a kernel (MTIRenderPipelineKernel, MTIComputePipelineKernel, or MTIMPSKernel) and apply it to your input images.

In Objective-C, you typically implement the <MTIFilter> protocol and provide a kernel class method that returns a pre-initialized kernel instance. The outputImage method then calls applyToInputImages:parameters:outputTextureDimensions:outputPixelFormat: on that kernel.

```objc
@interface MTIChromaKeyBlendFilter : NSObject <MTIFilter>
@property (nonatomic, strong, nullable) MTIImage *inputImage;
@property (nonatomic, strong, nullable) MTIImage *inputBackgroundImage;
@property (nonatomic) float thresholdSensitivity;
@property (nonatomic) float smoothing;
@property (nonatomic) MTIColor color;
@end

@implementation MTIChromaKeyBlendFilter

@synthesize outputPixelFormat = _outputPixelFormat;

+ (MTIRenderPipelineKernel *)kernel {
    static MTIRenderPipelineKernel *kernel;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        kernel = [[MTIRenderPipelineKernel alloc] initWithVertexFunctionDescriptor:[[MTIFunctionDescriptor alloc] initWithName:MTIFilterPassthroughVertexFunctionName] fragmentFunctionDescriptor:[[MTIFunctionDescriptor alloc] initWithName:@