pmndrs/xr

repository·main·Indexed 25 days ago

https://github.com/pmndrs/xr

A library to turn @react-three/fiber applications into interactive immersive XR (AR and VR) experiences. It includes @pmndrs/xr for XR state management via createXRStore, @pmndrs/pointer-events for enabling and filtering pointer events in Three.js scenes, and @pmndrs/handle for creating framework-agnostic handles to constrain object transformations (translation, rotation, and scaling).

Tokens
69.7K
Snippets
110
Records
344
Agent score
81%

What's inside pmndrs-xr

  1. Understand the role of the XR Store

    main

    The XR Store is the central component of all @react-three/xr experiences. It serves three primary purposes:

    1. Configuration: Allows you to set up the XR experience using a wide range of options.
    2. Control: Provides various functions to control the active XR experience.
    3. State Access: Provides access to the current state of the XR experience.
  2. Understand the XR Store in @react-three/xr

    main

    The XR store is the central component of @react-three/xr experiences. It serves three primary purposes:

    1. Configuration: Allows you to set up your XR experience using a wide range of options.
    2. Control: Provides various functions to manipulate and control the active XR experience.
    3. State Access: Provides access to the current state of the XR experience, allowing components to react to changes in the XR environment.
  3. Verify package compatibility for @react-three/xr

    main

    When setting up an XR project, ensure your dependencies align with the following peer requirements to avoid runtime or build errors:

    • @react-three/xr 6.x: Compatible with @react-three/fiber >=8, React >=18, and any version of three.
    • @react-three/drei 10.x: Requires React/React DOM 19.x and @react-three/fiber 9.x. Do not combine drei 10 with React 18 or fiber 8.
    • Vite: Keep Vite and @vitejs/plugin-react aligned. For example, @vitejs/plugin-react 6.x expects Vite 8.x; do not pair it with Vite 7.x.
    • TypeScript: If importing three or react-dom directly in strict TypeScript environments, include matching type packages: @types/three, @types/react, and @types/react-dom.
    • WebXR/IWER: Use a published iwer 2.x release for validation.
  4. Use the XROrigin component to control session origin

    main
    The XR session origin is a 3D transformation representing the user's position (typically the feet) when the session is recentered. Use the <XROrigin /> component to control this transformation and place it anywhere within your scene. Because it is a React component, you can attach it to moving objects or use it to apply global scale and rotation to the user's experience.
  5. Enable secondary input sources in XR store

    main

    By default, XR experiences typically only use primary inputs (one per hand, maximum of 2). To access additional tracked sources provided by standalone XR headsets (such as hand tracking alongside controller tracking), enable the secondaryInputSources flag when initializing the XR store via createXRStore.

    createXRStore({ secondaryInputSources: true })
  6. Implement teleportation using TeleportTarget and XROrigin

    main

    To implement teleportation, follow these steps:

    1. Manage Position State: Maintain a state (e.g., using useState) to track the user's current Vector3 position.
    2. Control User Origin: Pass the position state to the <XROrigin /> component to move the user's camera/origin.
    3. Define Teleport Targets: Wrap target meshes in a <TeleportTarget /> component. Use the onTeleport prop to receive the new position and update your state.

    Example implementation:

    const store = createXRStore({
      hand: { teleportPointer: true },
      controller: { teleportPointer: true },
    })
    
    export function App() {
      const [position, setPosition] = useState(new Vector3())
      return (
        <>
          <button onClick={() => store.enterVR()}>Enter VR</button>
          <Canvas>
            <XR store={store}>
              <ambientLight />
              <XROrigin position={position} />
              <TeleportTarget onTeleport={setPosition}>
                <mesh scale={[10, 1, 10]} position={[0, -0.5, 0]}>
                  <boxGeometry />
                  <meshBasicMaterial color="green" />
                </mesh>
              </TeleportTarget>
            </XR>
          </Canvas>
        </>
     )
    }
  7. Implement teleportation using XROrigin and TeleportTarget

    main

    To implement a teleportation system, follow these steps:

    1. Manage Position State: Use a state manager (like useState) to track the user's current Vector3 position.
    2. Control User Origin: Pass the position state to the <XROrigin /> component to move the user's camera/origin.
    3. Define Teleport Targets: Wrap your target meshes in a <TeleportTarget /> component.
    4. Handle Teleportation: Bind your position state setter to the onTeleport prop of <TeleportTarget />. This function is called whenever a user successfully teleports to that target.
    const store = createXRStore({
      hand: { teleportPointer: true },
      controller: { teleportPointer: true },
    })
    
    export function App() {
      const [position, setPosition] = useState(new Vector3())
      return (
        <>
          <button onClick={() => store.enterVR()}>Enter VR</button>
          <Canvas>
            <XR store={store}>
              <ambientLight />
              <XROrigin position={position} />
              <TeleportTarget onTeleport={setPosition}>
                <mesh scale={[10, 1, 10]} position={[0, -0.5, 0]}>
                  <boxGeometry />
                  <meshBasicMaterial color="green" />
                </mesh>
              </TeleportTarget>
            </XR>
          </Canvas>
        </>
     )
    }
  8. Improve Visual Quality in XR Applications

    main

    To ensure high visual quality and effective user experience in XR, follow these guidelines:

    Environment & Scene Construction:

    • Build domain-specific scenes rather than placeholder demos.
    • Include recognizable context like floors, walls, lanes, studios, or workstations.
    • Use multiple object types and readable scale cues.
    • Add lighting and material variation.

    Interactivity & Feedback:

    • Make interactive affordances obvious from plausible headset/controller poses.
    • Provide explicit feedback for hover, selection, hit, miss, completion, errors, score changes, or configuration changes.
    • Contextual Feedback Requirements:
      • Games: Show motion, progression, score, failure states, and final results.
      • Tools/Training: Show task state, current target, completion criteria, and final reports.
      • Simulations: Show telemetry, constraints, warnings, recovery, and mission outcomes.
      • Commerce: Show selected products/parts, variants, dimensions, price, and cart summaries.

    Camera & View Management:

    • Keep the primary object visible while UI is displayed. If a user looks at a menu, ensure the view returns to the active world object or frames both the panel and object.
    • Treat camera orientation as part of visual quality. After interacting with a control, move or turn the camera back so the next recorded frames show the active world object rather than an empty background.
  9. Avoid shell heredocs for code generation

    main
    When generating files (JSX, TypeScript, HTML, or vitexec), do not use shell heredocs such as cat <<EOF, cat > file <<EOF, or tee <<EOF. These methods frequently corrupt nested quotes, JSX self-closing tags, HTML doctype syntax, and JavaScript template strings. Instead, use patch or file-edit operations to ensure file integrity.
  10. Render high-quality videos using XRLayer

    main

    Use the XRLayer component to render videos with high performance and quality via the WebXR Layer API. This is ideal for preserving battery life and reducing latency for quad, cylinder, and equirect shapes. To use it, pass an HTML video element to the src prop of XRLayer.

    export function App() {
      const video = useMemo(() => {
        const result = document.createElement('video')
        result.src = 'test.mp4'
        return result
      }, [])
    
      return (
        <Canvas>
          <XR store={store}>
            <XRLayer position={[0, 1.5, -0.5]} onClick={() => video.play()} scale={0.5} src={video} />
          </XR>
        </Canvas>
      )
    }
  11. Render dynamic 3D scenes using XRLayer

    main

    Instead of static images or videos, XRLayer can host complete 3D scenes. Any React Three Fiber content placed as children of XRLayer will be rendered onto the layer. This content is re-rendered every frame, allowing for fully dynamic, high-quality visual content.

    <XRLayer position={[0, 1.5, -0.5]} scale={0.5}>
      <mesh>
        <boxGeometry />
        <meshBasicMaterial color="red" />
      </mesh>
    </XRLayer>