OpenXRS

To run the Vulkan example on Android devices (specifically Oculus Quest and Quest 2), you must set up the Android toolchain and specific shared libraries. The vulkan-android example uses the same source as the vulkan example but is configured for the Android runtime.

Prerequisites

1. Install cargo-apk.
2. Download Android SDK version 30 and set the $ANDROID_HOME environment variable to its path.
3. Download Android NDK version 23 and set the $ANDROID_NDK_ROOT environment variable to its path.

Library Setup

Before running, you must manually place specific libraries into the project structure:

1. Copy the arm64-v8a folder from ${ANDROID_NDK_ROOT}/sources/cxx-stl/llvm-libc++/libs/ into openxr/examples/libs/.
2. Obtain libopenxr_loader.so from the Oculus OpenXR Mobile SDK and place it in openxr/examples/libs/arm64-v8a.

Execution

Run the following command from the openxr directory:

cd openxr
cargo apk run --example vulkan-android

The HandTracker struct provides access to skeletal hand tracking data via the XR_EXT_hand_tracking extension. It is used to track the position and velocity of individual hand joints.

To use hand tracking, you must ensure the XR_EXT_hand_tracking extension is present in your session. The tracker is initialized for a specific hand (left or right) and provides access to a fixed number of joints defined by HAND_JOINT_COUNT.

Key types exported for hand tracking:

• Hand: Represents the hand being tracked (e.g., HandEXT).
• HandJoint: Represents a specific joint in the hand.
• HandJointLocation: Contains the spatial data for a joint.
• HandJointVelocity: Contains the velocity data for a joint.
• HandJointLocations: A type alias for an array of HandJointLocation for all joints.
• HandJointVelocities: A type alias for an array of HandJointVelocity for all joints.

use openxr as xr;

// Note: The actual creation of a HandTracker is currently internal (`pub(crate) fn create`), 
// but it requires a valid Session and a Hand type.
// Once obtained, you can access the raw handle via `.as_raw()`.

A FrameStream is a secondary interface to an OpenXR Session object that exposes only frame timing and synchronization operations (begin, end, and end_secondary). It is designed to provide a safe, synchronized interface for the presentation loop by using &mut self receivers, which statically guarantees that calls are made in the correct order as required by the OpenXR specification.

A typical presentation loop involves:

1. Waiting for the next frame via a FrameWaiter.
2. Acquiring an image from a Swapchain.
3. Calling frame_stream.begin() to indicate graphics work is starting.
4. Performing rendering/drawing.
5. Calling frame_stream.end() with the predicted display time and composition layers to submit the frame.
6. Releasing the swapchain image.

// Typical presentation loop body
let state = frame_waiter.wait().unwrap();
let image = swapchain.acquire_image().unwrap();
swapchain.wait_image(openxr::Duration::INFINITE).unwrap();

frame_stream.begin().unwrap();

if state.should_render {
    // draw scene...
}

// ... locate views and set matrices ...

swapchain.release_image().unwrap();
frame_stream
    .end(
        state.predicted_display_time,
        openxr::EnvironmentBlendMode::OPAQUE,
        &[&openxr::CompositionLayerProjection::new()...],
    )
    .unwrap();

The Swapchain<G> struct manages a set of images used for rendering to a XR session, where G is a type implementing the Graphics trait. It handles the lifecycle of swapchain images, including acquiring the next image for rendering and synchronizing with the compositor.

Image Lifecycle Workflow

To render to a swapchain, you must follow a specific sequence of calls to ensure proper synchronization with the compositor:

1. acquire_image(): Determines the index of the next image in the array to render to.
2. wait_image(timeout: Duration): Waits for the compositor to finish reading from the oldest unwaited acquired image. This must be called after acquire_image and before release_image.
3. release_image(): Releases the oldest acquired image. This must be called after wait_image and before the next acquire_image cycle.

Note: Calling wait_image before release_image has been called in the current cycle will trigger an assertion error.

// Conceptual workflow
let image_index = swapchain.acquire_image()?;

// ... perform rendering to image_index ...

swapchain.wait_image(Duration::from_millis(100))?;
swapchain.release_image()?;

On Android, OpenXR requires specific platform information to interface with the Android lifecycle and the Java VM. You must use AndroidPlatformInfo when calling Entry::linked, Entry::load, Entry::load_from, or Entry::create_instance.

Safety Warning: The arguments provided to AndroidPlatformInfo::new must be valid pointers:

• java_vm: A valid pointer to the JNI JavaVM, cast to *mut std::ffi::c_void.
• activity: A valid JNI reference to the android.app.Activity that drives the session lifecycle, cast to *mut std::ffi::c_void.

AndroidPlatformInfo handles the initialization of the XR_KHR_loader_init_android extension and the XR_KHR_android_create_instance extension internally.

// Safety: Ensure java_vm and activity are valid JNI pointers
let platform_info = unsafe { AndroidPlatformInfo::new(java_vm_ptr, activity_ptr) };

// Use platform_info to create the entry or instance
let entry = unsafe { Entry::load(&platform_info)? };