Spectacles Sample Templates and Examples

repository·main·Indexed 41 days ago

https://github.com/snapchat/spectacles-sample

A collection of templates and examples for building Lens experiences on Spectacles hardware, including the Agentic Playground AI assistant and the Spectacles UI Kit (UIKit). The UIKit provides interactive and interface components for Lens Studio, while the Agentic Playground demonstrates an agent-based architecture for real-time lecture summarization, conversational AI, and spatial awareness using the Remote Service Gateway.

Tokens
78.3K
Snippets
134
Records
389
Agent score
49%

What's inside Spectacles Sample

  1. Overview of the Material Library sample

    main

    The Material Library is an experimental sample project designed to collect and test various Materials specifically for the Spectacles platform. It serves as a reference for shader compatibility on Spectacles, including an "Essential" collection of materials and an "Advanced" collection (sourced from the Lens Studio Asset Library or the "Material Editor" sample).

    Note: This project is platform-specific and only works for Spectacles. Performance may be affected when using multiple materials simultaneously or rendering a large number of objects with the same material.

  2. Overview of the Fetch Sample Project

    main

    The Fetch sample project demonstrates the use of the Spectacles Fetch API. This project is specifically designed for the Spectacles platform and showcases how to interact with external data within a spatial environment.

    Note: This project will only work on the Spectacles platform.

  3. Overview of Shared Sync Controls

    main

    Shared Sync Controls is a sample project demonstrating a real-time collaborative AR experience for Spectacles. It showcases how two users can interact with shared UI elements—specifically a numeric value adjuster and RGB sliders—to control the color of a shared object in a scene.

    Key technologies used:

    • Spectacles Interaction Kit (SIK): Used for simplifying UI development on Spectacles.
    • Connected Lenses: Used to synchronize interactions and state across multiple devices in real-time.
  4. Overview of the Air Hockey Sample Project

    main

    The Air Hockey project is a sample Connected Lens designed specifically for the Spectacles platform. It demonstrates the use of the Spectacles Sync Kit to enable multiplayer functionality.

    Key features demonstrated include:

    • Connected Lenses: Multiplayer experiences for Spectacles.
    • Spectacles Sync Kit: Using sync entities and storage properties to synchronize game state across users.
  5. Overview of the Crop Sample

    main
    The Crop sample demonstrates how to 'crop' the physical environment using hand gestures on the Spectacles platform. It utilizes the camera module to access camera textures and intrinsics to align a digital crop with the physical camera view. This allows for spatial interaction where a user can define a capture area in 3D space.
  6. Overview of the Laser Pointer project

    main

    The Laser Pointer project is a demonstration of interactive pointers that collide with virtual objects in a shared spatial environment. It showcases how to create colored laser pointer marks that persist for a few seconds before fading.

    This project specifically demonstrates the integration of:

    • Spectacles Interaction Kit (SIK): For handling user interactions.
    • Spectacles Sync Kit (SSK): For enabling multiplayer synchronization, allowing multiple users to see and interact with the same objects and pointers in a shared space.
  7. Overview of Path Pioneer

    main
    Path Pioneer is a sample project designed for the Spectacles platform and Lens Studio. It demonstrates how to create and walk along a path using holographic reconstruction technology within a TypeScript codebase. The project allows users to design and store walkable paths (either as a line with a start/finish or a loop) and then experience running or walking along that path.
  8. Overview of Composite Camera Texture features

    main

    The Composite Camera Texture package is designed for combining and compositing multiple camera feeds and textures in Lens Studio. Key capabilities include:

    • Camera Compositing: Combining multiple camera feeds using various blend modes.
    • Real-time Processing: Efficiently compositing textures in real-time.
    • Custom Shaders: Utilizing shader-based operations for advanced compositing.
    • Layer Management: Organizing and controlling multiple texture layers.
  9. Overview of SnapML Chess Hints

    main
    SnapML Chess Hints is a Spectacles-exclusive project that uses computer vision to detect chess pieces on a physical board and provides move suggestions using AI. It demonstrates the integration of SnapML (machine learning), object tracking, and external AI APIs (like Google Gemini or StockFish) to enhance real-world activities.
  10. Overview of Agentic Playground features

    main

    Agentic Playground is an AI-powered educational assistant for Snap Spectacles. It uses a true agent architecture to provide:

    • Real-time Lecture Summarization: Captures and summarizes live lectures.
    • Intelligent Conversational AI: Contextual Q&A about lecture content.
    • Visual Diagram Generation: Creates mind maps and concept diagrams.
    • Spatial Awareness: Uses camera input to analyze the lecture environment.
    • Voice-First Interaction: Utilizes speech-to-text and text-to-speech.

    The system features a modular design with a clean separation of concerns and an extensible tool system that uses AI-powered tool routing instead of hard-coded rules.

  11. Overview of SnapML Starter for Spectacles

    main

    SnapML Starter is a template designed to demonstrate how to use custom machine learning models for object detection on Spectacles. It provides a complete end-to-end workflow for training custom YOLOv7 models and deploying them to create AR experiences that recognize and interact with real-world objects.

    Key capabilities include:

    • Object Detection: Real-time detection of multiple objects using custom YOLOv7 models.
    • 3D Object Placement: Using the PinholeCameraModel class to place 3D objects at the real-world locations of detected objects.
    • Smoothed Object Tracking: Maintaining stable AR effects on moving objects via configurable smoothing options.
  12. Overview of Spectacles Essentials project

    main

    The Spectacles Essentials project is a collection of foundational concepts and examples for creating lenses in Lens Studio specifically for deployment on Spectacles hardware. It utilizes TypeScript and JavaScript to demonstrate AR interaction and lens development.

    Key characteristics:

    • Platform Specific: This project is designed exclusively for the Spectacles platform.
    • Language Recommendation: While JavaScript is supported, the use of TypeScript is strongly recommended for better debugging and alignment with standard game engine patterns.
    • Use Case: It serves as a reference for foundational AR interaction patterns and can be used as context for LLMs to assist in personal project development.