Harbeth Documentation

Harbeth is a high-performance Swift library designed for GPU-accelerated real-time image processing, camera capture, and video processing on Apple platforms (iOS, macOS, tvOS, and watchOS). It serves as a GPU image and frame processing core, built on Metal technology and integrating with CoreImage and Metal Performance Shaders (MPS).

Core Capabilities:

• Input Sources: Processes MTLTexture, UIImage, NSImage, CIImage, CGImage, CMSampleBuffer, and CVPixelBuffer.
• Filter Ecosystem: Includes over 200+ built-in filters (color adjustment, blur, stylization, etc.).
• Customization: Supports custom Metal shaders and LUT-based filters (1D, 2D, 3D Cube, and Multi Zone Tables).
• Real-time Performance: Optimized for 60 FPS processing using Metal-powered rendering and automatic texture pooling.
• Integration: Designed to be embedded into existing pipelines like photo editors, camera previews, or video players. Note that Harbeth provides the processing core, while the host app manages camera sessions, recording, and export logic.

Harbeth provides a wide array of image and video processing filters categorized by their visual effect. Developers can choose from the following functional groups:

• Edge & Detail: High-accuracy edge detection (e.g., C7Canny, C7Sobel) and sharpening/clarity enhancements (e.g., C7Clarity, C7SharpenDetail).
• Distortion & Warp: Geometric and optical distortions (e.g., C7Bulge, C7Swirl, C7WaterRipple, C7Pixellated).
• Stylization: Artistic effects like oil painting (C7OilPainting), glitch art (C7Glitch, C7ShiftGlitch), and cartooning (C7Toon).
• Artistic, Sci-fi, Natural, Retro, Camera, and Scene Styles: Pre-set aesthetic filters ranging from C7PastelDream and C7Hologram to film simulations like C7FujiX100V and C7Polaroid.
• Matrix Processing: Low-level mathematical transformations using color matrices (C7ColorMatrix4x4) or convolution matrices (C7ConvolutionMatrix3x3).
• Utility: Tonal and color controls (e.g., C7ChromaKey, C7HighlightShadow, C7Levels) and skin smoothing (C7HighPassSkinSmoothing).
• Geometric Transform: Basic image manipulations like C7Crop, C7Flip, C7Resize, and C7Rotate.
• Generators & LUTs: Tools for creating new content (C7ColorGradient) or applying professional color grading via Look-Up Tables (C7ColorCube, C7LookupTable).

Harbeth is a high-performance image and video processing framework with the following key capabilities:

• Cross-Platform Support: Runs on iOS, macOS, tvOS, and watchOS. Supports UIKit, AppKit, and SwiftUI.
• Multi-Source Data Support: Processes MTLTexture, UIImage, NSImage, CIImage, CGImage, CMSampleBuffer, and CVPixelBuffer.
• High Performance: Uses Metal-driven rendering and Metal Performance Shaders (MPS) for real-time performance. Includes automatic texture pooling and memory management.
• Extensive Filter Ecosystem: Over 200+ built-in filters categorized by effect (Color, Blur, Edge, Distortion, Stylization, etc.).
• Customization: Supports custom filters via LUT (Lookup Tables), .cube files, or custom Metal shaders. Advanced multi-step effects can be created by inheriting from C7CombinationBase.
• Real-time & Video Processing: Supports real-time camera capture with filters and seamless video processing (local or network) using the integrated Kakapos library.
• Developer Experience: Provides a Swift-friendly API with linkable filter operations and operator overloading.

Harbeth optimizes GPU performance by adjusting threadgroup sizes based on the specific memory access pattern required by a filter. This ensures better utilization of Apple Silicon and Intel architectures through dynamic threadgroup sizing and memory coalescing.

Memory Access Patterns

Optimization Features

• Dynamic Threadgroup Sizing: Automatically adjusts based on GPU architecture and memory access pattern.
• Aspect Ratio Optimization: Adapts threadgroup shape to match texture dimensions for better memory coalescing.
• Architecture-Aware: Uses larger threadgroups on Apple Silicon for better utilization.
• Memory Coalescing: Optimizes memory access patterns to reduce GPU memory latency.

HarbethView is a native SwiftUI component that allows you to display filtered images within a SwiftUI view hierarchy. It supports state-driven updates, meaning if you pass a filter parameter tied to a @State variable, the view will automatically re-render with the new effect.

Best Practices

• Use @State for filter parameters to enable real-time adjustments.
• Wrap HarbethView in VStack or ZStack for complex layouts.
• Apply standard SwiftUI modifiers (like .cornerRadius() or .shadow()) directly to the output of the HarbethView closure.

import SwiftUI
import Harbeth

struct FilteredImageView: View {
    @State private var inputImage: UIImage = UIImage(named: "sample")!
    @State private var intensity: Float = 0.5

    var body: some View {
        VStack {
            HarbethView(image: inputImage, filters: [
                CIHighlight(highlight: intensity),
                C7WaterRipple(ripple: intensity),
            ]) { image in
                image
                    .resizable()
                    .aspectRatio(contentMode: .fit)
                    .cornerRadius(12)
                    .shadow(radius: 5)
            }

            Slider(value: $intensity, in: 0...1)
                .padding()
        }
        .padding()
    }
}

Harbeth supports macOS with native AppKit integration. You can use HarbethIO to process NSImage objects within an NSViewController.

Implementation Pattern

1. Create a filter chain using types conforming to C7FilterProtocol.
2. Initialize HarbethIO with the NSImage and the filter array.
3. Use transmitOutput with a completion handler to update the NSImageView on the main thread.

import Cocoa
import Harbeth

class ImageProcessingViewController: NSViewController {
    @IBOutlet weak var imageView: NSImageView!

    func applyFilter(to image: NSImage) {
        let filters: [C7FilterProtocol] = [
            C7Brightness(brightness: 0.1),
            C7Contrast(contrast: 1.2),
            C7Saturation(saturation: 1.1)
        ]

        let dest = HarbethIO(element: image, filters: filters)

        dest.transmitOutput { [weak self] result in
            switch result {
            case .success(let output):
                DispatchQueue.main.async {
                    self?.imageView.image = output
                }
            case .failure(let error):
                print("Filter application failed: \(error)")
            }
        }
    }
}