WebXR First Steps Tutorial

repository·main·Indexed 41 days ago

https://github.com/meta-quest/webxr-first-steps

A 2-hour hands-on tutorial for building immersive WebXR experiences using Three.js. The guide covers creating a target-practice game with VR controller support, animations, 3D models, and hit detection. It includes instructions for local development with Node.js 20.x+, deploying via GitHub Pages, and using the IWER (Immersive Web Emulation Runtime) for browser-based emulation.

Tokens
9.1K
Snippets
30
Records
35
Agent score
52%

What's inside WebXR First Steps

  1. Implement proximity-based hit detection

    main

    For simple game mechanics where targets are relatively uniform, you can use a proximity-based hit test. This involves iterating through active projectiles (bullets) and checking the distance to visible targets using target.position.distanceTo(bullet.position).

    If the distance is below a certain threshold (e.g., 1 unit), a hit is registered. Common actions upon a hit include:

    1. Removing the bullet from the scene.
    2. Hiding the target temporarily.
    3. Respawning the target at a new random position after a delay.
    4. Incrementing the score and updating the UI.

    Note: For irregularly shaped objects or more precise requirements, use raycasting instead of proximity checks.

    function onFrame() {
    	// ... unchanged params
    	Object.values(bullets).forEach((bullet) => {
    		// ... unchanged logic
    		targets
    			.filter((target) => target.visible)
    			.forEach((target) => {
    				const distance = target.position.distanceTo(bullet.position);
    				if (distance < 1) {
    					delete bullets[bullet.uuid];
    					scene.remove(bullet);
    
    					// make target disappear, and then reappear at a different place after 2 seconds
    					target.visible = false;
    					setTimeout(() => {
    						target.visible = true;
    						target.position.x = Math.random() * 10 - 5;
    						target.position.z = -Math.random() * 5 - 5;
    					}, 2000);
    
    					score += 10; // Update the score when a target is hit
    					updateScoreDisplay(); // Update the rendered troika-three-text object
    				}
    				});
    		});
    }
  2. How meshes, geometries, and materials work in Three.js

    main

    In Three.js, a mesh is the fundamental object used to render 3D shapes. It is composed of two distinct parts:

    1. Geometry: Defines the mathematical shape or structure of the object (e.g., PlaneGeometry, BoxGeometry, SphereGeometry).
    2. Material: Defines the visual appearance, such as color and how it reacts to light (e.g., MeshStandardMaterial).

    To create an object, you instantiate a THREE.Mesh by passing both the geometry and the material as arguments, then add that mesh to your scene using scene.add(mesh).

    // 1. Define Geometry
    const floorGeometry = new THREE.PlaneGeometry(6, 6);
    
    // 2. Define Material
    const floorMaterial = new THREE.MeshStandardMaterial({ color: 'white' });
    
    // 3. Combine into a Mesh
    const floor = new THREE.Mesh(floorGeometry, floorMaterial);
    
    // 4. Add to scene
    scene.add(floor);
  3. Efficiently spawn objects using prototypes

    main

    To create multiple instances of an object (like bullets) efficiently, define a bulletPrototype using a single THREE.Geometry and THREE.Material. Instead of creating new geometry for every instance, use the .clone() method on the prototype mesh. This allows all instances to share the same underlying geometry and material data, significantly reducing memory usage.

    const bulletGeometry = new THREE.SphereGeometry(0.02);
    const bulletMaterial = new THREE.MeshStandardMaterial({ color: 'gray' });
    const bulletPrototype = new THREE.Mesh(bulletGeometry, bulletMaterial);
    
    // To spawn:
    const newBullet = bulletPrototype.clone();
    scene.add(newBullet);
  4. Use the built-in WebXR emulator

    main

    The project includes a built-in emulation setup in ./src/init.js using IWER (Immersive Web Emulation Runtime) and @iwer/devui.

    How it works:

    • It automatically activates if the browser lacks native WebXR support.
    • @iwer/devui provides a developer interface to manipulate headset and controller transforms and simulate input events.
    • Play Mode: A button on the interface top bar that locks your cursor, allowing you to control the emulated XR device like a first-person 3D experience on a PC.

    Note: If you use the Immersive Web Emulator browser extension, it will not conflict. However, if you use other WebXR emulators, it is recommended to disable them to avoid conflicts with the built-in IWER setup.

  5. Build and deploy the WebXR application

    main

    Deploying via GitHub Pages

    This project uses a GitHub Actions workflow (.github/workflows/deploy.yml) to automate builds and deployments to GitHub Pages.

    To enable:

    1. Fork the repository to your GitHub account.
    2. Go to Settings > Pages.
    3. Under Build and Deployment, set the Source to GitHub Actions.

    Every push to the main branch will now trigger an automatic deployment.

    Manual Build for Custom Hosting

    To host the app on your own solution, generate the static files manually:

    1. Run npm run build.
    2. Upload the contents of the generated dist folder to your hosting provider.
    npm run build
  6. Handle VR controller input and button presses

    main

    To interact with VR controllers, you can check the controllers object within the onFrame loop. The controllers object provides access to individual controllers (e.g., controllers.right) which contain a gamepad and a raySpace.

    Detecting Button Presses

    Use the gamepad.getButtonClick(button) method from the gamepad-wrapper library to detect single button presses. This method tracks state frame-by-frame to ensure a single press is registered once.

    Spawning Objects at Controller Position

    When spawning an object (like a bullet) from a controller, you must align it with the controller's raySpace. Since spawned objects are typically added directly to the scene, you must manually retrieve and apply the controller's world transform:

    1. Position: Use raySpace.getWorldPosition(targetPosition).
    2. Orientation: Use raySpace.getWorldQuaternion(targetQuaternion).

    Note: raySpace represents the XR controller's position and direction in 3D space.

    import { XR_BUTTONS } from 'gamepad-wrapper';
    
    function onFrame(
    	delta,
    	time,
    	{ scene, camera, renderer, player, controllers },
    ) {
    	if (controllers.right) {
    		const { gamepad, raySpace } = controllers.right;
    		if (gamepad.getButtonClick(XR_BUTTONS.TRIGGER)) {
    			// 1. Clone a prototype object
    			const bullet = bulletPrototype.clone();
    			
    			// 2. Add to scene
    			scene.add(bullet);
    			
    			// 3. Align with controller's world transform
    			raySpace.getWorldPosition(bullet.position);
    			raySpace.getWorldQuaternion(bullet.quaternion);
    		}
    	}
    }
  7. Configure client-specific agentic setups

    main

    The repository provides specific configuration files and installation commands for various AI coding agents and IDEs. Use the following instructions based on your preferred client:

    ClientRecommended Setup
    VS CodeInstall meta.meta-vr-dev; optionally run npx -y @meta-quest/hzdb mcp install vscode
    CursorInstall meta.meta-vr-dev; optionally run npx -y @meta-quest/hzdb mcp install cursor
    Claude Code/plugin marketplace add meta-quest/agentic-tools; /plugin install meta-vr@meta-quest
    Gemini CLIgemini extensions install https://github.com/meta-quest/agentic-tools
    GitHub CopilotUse the repository instructions and install the recommended VS Code extension
    ClineUse AGENTS.md; configure MCP if supported
    Roo CodeUse AGENTS.md; configure MCP if supported
    Windsurfnpx -y @meta-quest/hzdb mcp install windsurf
    OpenCodenpx -y @meta-quest/hzdb mcp install open-code
    Codexnpx -y @meta-quest/hzdb mcp install codex
  8. Set up positional audio with Three.js

    main

    To implement spatialized audio in a WebXR scene, you must attach an AudioListener to the camera and use PositionalAudio objects for sounds that should originate from specific 3D coordinates. Attach the audio objects to the relevant 3D meshes (e.g., a blaster or a scoreboard) so the sound follows them in the scene.

    // 1. Create listener and attach to camera
    const listener = new THREE.AudioListener();
    camera.add(listener);
    
    const audioLoader = new THREE.AudioLoader();
    
    // 2. Create positional audio for a specific object (e.g., laserSound)
    const laserSound = new THREE.PositionalAudio(listener);
    audioLoader.load('assets/laser.ogg', (buffer) => {
        laserSound.setBuffer(buffer);
        blasterGroup.add(laserSound);
    });
    
    // 3. Create positional audio for another object (e.g., scoreSound)
    const scoreSound = new THREE.PositionalAudio(listener);
    audioLoader.load('assets/score.ogg', (buffer) => {
        scoreSound.setBuffer(buffer);
        scoreText.add(scoreSound);
    });
  9. Follow the WebXR tutorial chapters

    main

    The tutorial is structured into six progressive chapters:

    1. Chapter 1: Creating Simple Objects: Adding 3D shapes (cubes, spheres, cones).
    2. Chapter 2: Interacting with Controllers: Implementing VR controller interactions and user input.
    3. Chapter 3: Animating Objects: Time-based animation and moving bullets.
    4. Chapter 4: Loading GLTF Models: Using complex 3D models instead of simple geometries.
    5. Chapter 5: Hit Detection and Score Tracking: Proximity-based detection and score displays.
    6. Chapter 6: Finishing Touches: Adding audio, haptics, and visual effects.
  10. Set up bullet velocity and lifespan on fire

    main

    When a user triggers a shot (e.g., via XR_BUTTONS.TRIGGER), clone a bullet prototype, position it at the controller's raySpace, and calculate its trajectory.

    To ensure the bullet moves in the direction the controller is pointing, take a forwardVector and apply the controller's world quaternion to it. Store the resulting velocity and the remaining lifespan in the Three.js userData property of the bullet object.

    if (controllers.right) {
    	const { gamepad, raySpace } = controllers.right;
    	if (gamepad.getButtonClick(XR_BUTTONS.TRIGGER)) {
    		const bullet = bulletPrototype.clone();
    		scene.add(bullet);
    		raySpace.getWorldPosition(bullet.position);
    		raySpace.getWorldQuaternion(bullet.quaternion);
    
    		const directionVector = forwardVector
    			.clone()
    			.applyQuaternion(bullet.quaternion);
    		bullet.userData = {
    			velocity: directionVector.multiplyScalar(bulletSpeed),
    			timeToLive: bulletTimeToLive,
    		};
    		bullets[bullet.uuid] = bullet;
    	}
    }