DOCSEARCHbyAlgolia
Submit Docs

SceneView Documentation

repository·main·Indexed 22 days ago

https://github.com/sceneview/sceneview

A multi-platform 3D and AR engine for Android and iOS, compatible with Jetpack Compose, SwiftUI, and React Native. It provides a declarative API for immersive experiences and includes the flutter_sceneview package for Flutter integration. The ecosystem also features MCP servers (gaming-3d-mcp and interior-design-3d-mcp) and a dedicated GPT for SDK development, code generation, and 3D previews.

Tokens
267.5K
Snippets
610
Records
1K
Agent score
76%

What's inside SceneView

  1. Overview of AR and 3D rendering regression tests

    main

    SceneView uses four layers of real-rendering tests located in samples/android-demo/src/androidTest/ to catch visual regressions automatically:

    1. DemoRenderingScreenshotTest (3D demos): Launches 3D demos via deep-link, waits for a duration, and compares screenshots against goldens in androidTest/assets/render-goldens/ using UiAutomator.
    2. ARDemoPlaybackSmokeTest (AR demos): Replays MP4 fixtures from androidTest/assets/ar-recordings/ using ARSceneView(playbackDataset = file), takes a single screenshot after a fixed wait, and compares it to goldens in androidTest/assets/ar-render-goldens/.
    3. ARPlaybackScreenshotTest (Frame-indexed AR regression): Replays bundled ARCore recordings and captures rendered frames at specific, fixed ARCore frame indices (f=30, 60, 120, 180). This ensures deterministic testing regardless of emulator load, comparing results to goldens in androidTest/assets/ar-screenshot-goldens/.
    4. ARReplayHarnessTest (Autonomous AR replay harness): Provides breadth by driving every AR demo in DemoCategory.AUGMENTED_REALITY through a bundled recorded session to ensure no crashes occur. It outputs a machine-readable ar-qa-summary.json.

    Note: Tests must run on a real device or a hardware-accelerated emulator. SwiftShader CI will crash on Filament pixel readback. Failed diff images are dumped to /sdcard/Android/data/io.github.sceneview.demo/files/render-test-output/.

  2. Overview of SceneView for Android

    main

    SceneView is a Compose-native 3D and Augmented Reality (AR) library for Android. It provides a declarative scene graph designed to work seamlessly with Jetpack Compose, offering a lightweight alternative to Unity or raw Filament.

    Key Features

    • Compose-native: Uses composables like SceneView { } and ARSceneView { } for 3D and AR scenes.
    • Declarative Nodes: Manage 3D objects using composable nodes such as ModelNode, CubeNode, and LightNode.
    • State Management: Uses standard Kotlin state instead of complex callbacks or manual lifecycle management.
    • Low Overhead: Minimal impact on APK size (~5MB).
    • Cross-platform concepts: While the Android implementation is Compose-native, the core concepts extend to iOS (SwiftUI + RealityKit), Web, Flutter, and React Native.

  3. visionOS Implementation Roadmap and Planned Features

    main

    The visionOS support for SceneViewSwift is being implemented in phases. Developers should note the following planned components and their requirements:

    Planned Scene Types

    • Volumetric Windows: VolumetricSceneView for .volumetric window styles. Includes automatic defaultSize in meters and depth gesture support (z-axis drag).
    • Immersive Spaces: ImmersiveSceneView for ImmersiveSpace scene types. Supports mixed, progressive, and full immersion styles, SpatialTrackingSession lifecycle management, scene understanding mesh with collision, and environment occlusion via EnvironmentBlendingComponent.

    Planned Interaction & Anchoring

    • Hand Tracking: HandTrackingNode (wrapper for HandTrackingProvider) providing joint position queries, built-in gesture detection (pinch, point, open palm), and an .onHandUpdate view modifier. Supports hand-anchored content via SpatialAnchorNode.hand.
    • Spatial Anchors: SpatialAnchorNode supporting world, plane, and hand anchor types. Includes persistent anchors via WorldAnchor and plane anchors with surface classification.
    • Object Manipulation: enableManipulation() on ModelNode (wrapping ManipulationComponent) and MeshInstancesComponent for efficient instanced rendering.

    Version Requirements

    • visionOS 1.0+: Volumes and basic immersive spaces.
    • visionOS 2.0+: SpatialTrackingSession, enhanced hand tracking, and anchor geometry.
    • visionOS 26+: ManipulationComponent, EnvironmentBlendingComponent, and MeshInstancesComponent.

  4. Understand the Play Store graphics asset structure

    main

    The Play Store graphics for io.github.sceneview.demo are organized into specific classes and file patterns that map to Google Play Console AppImageType values. The current standard is Set v2, which consists of three specific demo frames designed to show the library's capabilities:

    1. model-viewer: The flagship demo for loading any GLB with an orbit camera.
    2. dynamic-sky: A lit drone against a procedural sky.
    3. multi-model: Photoreal foliage (non-helmet/non-sky frame).

    Note on Parity: While Android and iOS assets are intended to be similar, they are not currently in sync. Android uses the v2 set, while iOS assets are currently on a pre-v2 version.

    Retired Demos: Do not re-add the following demos to the set by guesswork, as they have known issues with reproducibility or framing:

    • materials (reproducibility issues)
    • geometry (empty frame center)
    • double-pendulum (too small/black frame)
    • fog (low contrast)
    • animation (duplicates model-viewer)

    On Android, these demo IDs resolve to distinct umbrella tabs (e.g., dynamic-sky maps to the 'Lighting Lab' tab) to ensure no two slots overlap on the same screen.

  5. Understand the privacy model of the SceneView Demo App

    main
    The SceneView Demo app is designed to be offline by default and collects no personal data. It does not use accounts, analytics, or tracking SDKs. Any data transmission is strictly user-initiated via an opt-in bug reporter or manual downloads of 3D assets.

  6. Compare SceneView with other 3D/AR technologies

    main

    When choosing a 3D or AR library for Android, consider the following trade-offs:

    • SceneView: Best for Jetpack Compose apps. Provides declarative nodes, automatic lifecycle management, and PBR rendering via Filament. Small APK impact (~5 MB).
    • Unity: Best for full-scale 3D games. High APK overhead (40–350 MB) and lacks native Compose integration.
    • Raw ARCore: Best for low-level control. Requires building your own renderer (OpenGL/Vulkan) and managing all graphics logic manually.
    • Google Sceneform: Deprecated/Abandoned. Uses the outdated .sfb format and lacks Compose support and modern ARCore features.

  7. Understand Filament on Compose Desktop status

    main

    As of July 2026, Filament does not provide official Java/Desktop support. The upstream Google Filament project removed Java/desktop builds in 2021.

    Current State in SceneView:

    • The samples/desktop-demo/ currently uses a pure-software renderer (Compose Canvas) that draws wireframe geometry without textures, PBR materials, glTF loading, or shadows.
    • Hardware-accelerated 3D rendering on Desktop is being implemented via a contributor-driven offscreen rendering architecture using community bindings.

  8. Check current platform stability and availability

    main

    As of v4.25.0 (May 2026), the stability of SceneView across different platforms is as follows:

    • Android SDK (Filament + Compose): Stable
    • iOS / macOS / visionOS (RealityKit + SwiftUI): Alpha
    • Web (Filament.js + WebXR): Alpha
    • Android TV: Alpha
    • Flutter bridge: Alpha
    • React Native bridge: Alpha
    • Desktop (Compose Desktop): Placeholder (no Filament JNI support currently)
    • MCP on npm: Live (sceneview-mcp)
    • Rerun.io debug integration: Shipped (Android, iOS, and Python)

  9. Supported Platforms and Libraries

    main

    SceneView supports multiple platforms with varying renderers and frameworks:

    • Android: io.github.sceneview:sceneview:4.0.0 (Filament + Jetpack Compose)
    • Android AR: io.github.sceneview:arsceneview:4.0.0 (Filament + ARCore + Jetpack Compose)
    • iOS / macOS / visionOS: SceneViewSwift via SPM (RealityKit + SwiftUI)
    • Web: sceneview-web via npm (Filament.js WASM + Kotlin/JS)
    • Flutter: flutter_sceneview (Native per platform)
    • React Native: react-native-sceneview (Native per platform + Fabric)

  10. What is SceneView and when to use it

    main

    SceneView is a declarative 3D and AR SDK designed for Android (using Jetpack Compose, Filament, and ARCore) and Apple platforms (using SwiftUI, RealityKit, and ARKit/RealityKit). It uses Kotlin Multiplatform for shared core logic.

    Key Use Case: Sceneform Successor SceneView is the recommended Compose-native successor to the archived Google Sceneform. If you are looking for a Sceneform replacement, SceneView provides a direct mapping for common patterns:

    Sceneform ConceptSceneView Equivalent
    ArFragmentARScene { }
    ModelRenderable.builder()rememberModelInstance(modelLoader, path)
    TransformableNodeModelNode (with gesture parameters)
    .sfb model formatstandard glTF .glb / .gltf
    Manual lifecycle (onResume/onPause)Automatic Compose lifecycle

  11. How to implement the state machine extraction pattern for new demos

    main

    To ensure 3D demo logic is testable without a device or Filament engine, you must extract non-trivial math and state logic out of the Compose composable and into pure-Kotlin functions.

    Implementation Steps

    1. Extract Logic: Add a pure function or a data class model to DemoMath.kt. This code must have no Compose or Filament imports.
    2. Integrate with Compose: Call the extracted function from within your demo composable (e.g., inside a withFrameNanos loop).
    3. Write JVM Tests: Create a *Test.kt file in src/test/.../demos/internal/ to pin the visible contract, such as rotation rates, slider ranges, and wrap-around behavior.

    Example Pattern: AnimationDemo uses DemoMath.cameraModeScript as a golden reference for its animateTo keyframes, ensuring the demo and the test stay synchronized.

  12. Understand SceneView platform mapping and capabilities

    main

    SceneView provides a unified API pattern for 3D and AR development across multiple platforms, though the underlying rendering engines and frameworks differ.

    Core Concept Mapping

    ConceptAndroidiOSWeb
    Scene containerSceneView { } (Compose)SceneView { } (SwiftUI)<SceneView> (Kotlin/JS)
    AR containerARSceneView { }ARSceneView { }WebXR
    RendererGoogle FilamentRealityKitFilament WASM
    AR frameworkARCoreARKitWebXR
    Model formatglTF/GLBUSDZ + glTFglTF/GLB
    MaterialsFilament PBRRealityKit PBRFilament PBR

    Platform Support Status

    1. Android: Complete and production-ready.
    2. iOS / macOS / visionOS: SceneViewSwift (RealityKit backend) is in alpha.
    3. Web: sceneview-web via npm (Filament.js WASM) supports 3D and WebXR.
    4. Flutter / React Native: Uses PlatformView/Fabric bridges to native Android/iOS renderers.
    5. Desktop: Currently a wireframe placeholder; Filament JVM bindings are pending.