Unity Passthrough Camera API Samples

The Unity-PassthroughCameraAPISamples project demonstrates how to access Quest camera data using the PassthroughCameraAccess component from the Mixed Reality Utility Kit (MRUK).

Key capabilities provided by this component include:

• Precise timestamps for improved camera-to-world alignment.
• Simultaneous access to both left and right headset cameras.
• Complete camera metadata, including intrinsics, extrinsics, and pose information.

Follow these steps to run the provided samples:

1. Clone the repository and open the project in Unity 6000.0.38f1 or newer.
2. Navigate to the Assets/PassthroughCameraApiSamples/ folder to find sample scenes.
3. Use the built-in tool to ensure your project is configured correctly: Meta > Tools > Project Setup Tool.
4. Build and deploy the project to a Quest 3 or Quest 3S device.

Before cloning, ensure you have Git LFS installed to handle large files correctly.

1. Install Git LFS:

git lfs install

2. Clone the repository:

git clone https://github.com/oculus-samples/Unity-PassthroughCameraApiSamples

To use this project, ensure your environment meets the following specifications:

Software Requirements

• Unity: 6000.0.38f1 or newer
• Meta MRUK: v81 or higher
• Unity Inference Engine: v2.2.1 (specifically required for the MultiObjectDetection sample)

Hardware & OS Requirements

• Hardware: Quest 3 or Quest 3S
• OS: Horizon OS v74 or higher

Configuration & Permissions

• Permissions: You must include horizonos.permission.HEADSET_CAMERA in your project.
• Passthrough: Passthrough must be enabled in your project settings.

NOTE

Previewing: You must use a physical headset or Meta Horizon Link v2.1 or later to preview the passthrough camera. The XR Simulator does not support the Passthrough Camera API.

The project includes five distinct sample scenes located in Assets/PassthroughCameraApiSamples/:

• CameraViewer: Displays a 2D canvas containing the camera feed.
• CameraToWorld: Demonstrates aligning RGB camera pose with Passthrough and transforming 2D coordinates into 3D world space rays.
• BrightnessEstimation: Shows how to adapt experiences based on the detected environment brightness.
• MultiObjectDetection: Demonstrates real-world object recognition using the Unity Inference Engine.
• ShaderSample: Shows how to apply custom GPU effects to the camera texture.

Additional Components:

• PassthroughCamera: Contains the core C# classes and utilities for camera access.
• StartScene: A menu scene used to switch between the different samples.