OpenReel Video Documentation

The Bezier Pen + Animatable Path System provides vertex-preserving path animation, a pen tool for drawing with click-drag handles, on-canvas vertex editing, bezier masks, and keyframed mask-path animation (suitable for rotoscoping). The system uses a shared vertex model for both shapes and masks.

Core Architecture Components:

• MotionShapePathPoint: The single vertex model containing x/y coordinates and in/out handles.
• interpolateMotionPathPoints: An interpolator used for equal-count lerp of vertices and handles (with a resample fallback).
• stage-path-editing.ts: A helper for web-based pen draw and edit interactions.
• MasksPanel: Used for mask drawing.
• pathKeyframes: Evaluated using the same interpolator as the path points.

The Expression Engine is designed to provide AE-grade (After Effects grade) expression rigging. Key capabilities include:

• Multi-line expressions: Support for complex, multi-line logic.
• Time-based evaluation: Access to valueAtTime for temporal control.
• Cross-layer references: Cycle-guarded references using thisComp.layer() and thisLayer.
• Keyframe introspection: Access to keyframe data via key, numKeys, and nearestKey.
• Effect access: Accessing parameters via effect(name)(param).
• Expression-control effects: Specialized Slider, Checkbox, and Angle effects for controlling expressions.
• Developer/User tooling: Visible error surfacing and a link-picker (pick-whip) for easier referencing.

OpenReel v2 is a creator platform designed to function fully offline using on-device AI (Core ML), while offering cloud-based GPU enhancements for heavy workloads. The ecosystem integrates a token-based economy ($ORC on Solana), a creator marketplace, and real-time multiplayer collaboration.

Key Architecture Components:

• iOS App: Handles local editing, on-device AI, and local export.
• Cloudflare Workers: Acts as the API Gateway for auth, marketplace, project sync, and AI routing.
• AWS Spot GPU Fleet: Executes heavy AI tasks (upscaling, generation) and premium renders.
• Solana: Manages the $ORC token, marketplace escrow, and creator earnings.

The Paper Shaders integration aims to incorporate the Paper Design shader catalog (~29 shaders) as first-class presets within the Motion Creator and as clip shader-effects in the video editor.

Key architectural components include:

• MotionShaderRenderer Extension: Supports type-driven uniform uploads (including vec4 colors, arrays, textures, custom vertex shaders, static sizing uniforms, and time scale).
• Paper Adapter Module: A data-driven module that maps Paper fragment sources to MotionShaderDef objects.
• Video Editor Integration: Implements an Effect.type:"shader" branch that utilizes the extended renderer.

Technical Requirements:

• Language: TypeScript (strict mode)
• Graphics API: WebGL2 #version 300 es
• Dependencies: @paper-design/shaders@0.0.76 (required by packages/core)

The v2.0 implementation introduces on-device AI features for the iOS app using Core ML, allowing features to run locally without an internet connection.

Key AI Features:

• Auto-captions
• Smart cut
• Background removal
• Auto-reframe
• Scene detection
• Audio enhancement
• Smart color match
• Object tracking

Architecture Pattern: Each AI feature is implemented as a standalone service located in Core/AI/. These services consume inputs (such as video frames or audio buffers) and produce specific outputs (such as subtitles, masks, or crop rectangles). A centralized ModelManager is responsible for the lifecycle of Core ML models, including downloading, caching, and on-demand loading.

The Filter Presets v1 system provides a curated catalog of ~60 LUT-based color grades for mobile clients (iOS and Android). The architecture consists of three layers:

1. Build-time Toolchain: A Python tool that reads YAML recipes and emits .cube LUTs and a manifest.json.
2. Hosting Layer: Cloudflare R2 hosts the filter bundle behind a public custom domain (filters.openreel.video). It uses native R2 serving with ETag and immutable caching (no Worker proxy required).
3. Client Layer: Mobile clients (iOS Swift and Android Kotlin) fetch the manifest and lazy-load LUTs on demand using sha256 integrity checks. Rendering is handled via CIColorCube on iOS and Media3 GlEffect with GL_TEXTURE_3D on Android.

When applying a filter, the clip.filter field (type AppliedFilter?) is used to carry the filter id and intensity (0–100%).

The Studio redesign moves from a node-graph editor to a subject+behavior model.

Core Data Model:

• Scene: Defined as { subjects: Subject[], sampleClipId }.
• Subjects: Can be of type face, subject_silhouette, or full_frame.
• Behaviors: A list of preset recipes or atomic blocks attached to subjects.

Key Technical Details:

• Graph Derivation: The existing @openreel/fxpkg Graph is derived from the Scene via sceneToGraph(scene) at preview-compile/publish time.
• Preview: Uses MediaPipe Tasks Vision (FaceLandmarker, ImageSegmenter) on real video.
• Tech Stack: React 18 (using useReducer + Context), Vite 5, Tailwind 3, and TypeScript strict mode.

The Aurora engine is a native, physically-based renderer designed to be integrated into the OpenReel desktop application. It follows a hybrid architecture to balance development speed and performance:

• Final Rendering: Uses an embedded Cycles path tracer (C++) to achieve Blender-class quality (global illumination, volumetrics, ACES color, etc.).
• Real-time Viewport: A separate real-time PBR viewport (intended to be built with Rust + wgpu) to provide instant editor feedback, eventually replacing the existing Three.js scrub-preview.
• Integration Pattern: The engine is implemented as a sidecar render process. Electron communicates with this native process via IPC. This prevents long GPU renders from blocking the UI and allows the same binary to be used on cloud GPU workers.

The Inspector in @openreel/web has transitioned from a single long-scrolling column using scroll anchors to a Real Tabbed Inspector.

Key architectural changes:

• Navigation: Uses horizontal top tabs (icon + label) that swap the panel content to show only the controls for the active tab.
• Persistence: The active tab ID is synced to the ui-store via the inspectorActiveTab key, ensuring the chosen tab is preserved when switching between clips of the same type.
• Composition: The inspector is now a composition of specialized components: ClipInspector manages state and tab derivation, while individual *Tab.tsx components wrap existing section components inside InspectorGroup primitives.
• Error Handling: Each tab's body is wrapped in an InspectorTabErrorBoundary to prevent a single failing section from crashing the entire inspector panel.