Project Aria Documentation Archive

website·Indexed 30 days ago

https://lizargall.github.io

An archive of technical writing for Project Aria, featuring documentation on the Aria Client SDK, TICSync, Aria Research Kit (ARK), and MPS. Includes software release notes for Mobile Companion App v170, eye gaze calibration, MPS CLI guides, and processing overviews.

Tokens
18.9K
Snippets
41
Records
154
Agent score
75%

What's inside Project Aria

  1. Overview of Project Aria research platform

    Project Aria is a research platform designed to accelerate egocentric (human perspective) AI research in fields such as computer vision, robotics, and contextual AI. It provides tools, data, and resources through Open Science Initiatives (OSI) and the Aria Research Kit (ARK).
  2. Overview of Project Aria MPS CLI

    The Project Aria Machine Perceptions Services Command Line Interface (MPS CLI) is a tool used to request and receive Machine Perception Services (MPS). It offers more features and more robust file uploading than the Desktop Companion app. The CLI supports two primary modes:

    • Single Mode: Processes each recording individually. Output is saved in the same directory as the input VRS file. This is the most common usage.
    • Multi Mode: Processes multiple recordings together to generate multi-sequence SLAM data in a shared coordinate frame. Requires a specified output directory.

    Inputs can be individual files or directories, and multiple inputs can be processed in a single command.

  3. Overview of Project Aria TICSync

    TICSync allows users to capture time-synchronized data between multiple Project Aria glasses on a single Wi-Fi network. One pair of glasses acts as a server (Wi-Fi access point), and all other glasses send time requests to this leader device to create a synchronized time reference. The method typically achieves an accuracy of better than 1ms after a warm-up period of approximately 45 seconds.
  4. Overview of Project Aria and Aria Data Tools

    Project Aria is a research device designed to collect first-person view (egocentric) data, including video, audio, eye-gaze, inertial, and location information, to advance machine perception and AI research for AR glasses. Aria Data Tools is a suite of C++ and Python3 tools used to read and visualize Project Aria sequences and sensor data, as well as retrieve calibration data and interact with camera models.
  5. Overview of Project Aria Eye Gaze Data Outputs

    Project Aria's Machine Perception Services (MPS) estimate gaze direction using Eye Tracking (ET) camera images. The estimation is performed in the Central Pupil Frame (CPF). Depending on the recording and calibration, MPS produces the following output files:

    • summary.json: A high-level report on the eye gaze generation process.
    • general_eye_gaze.csv: Gaze data based on the standard eye gaze configuration (available for all recordings with ET cameras enabled).
    • personalized_eye_gaze.csv: Gaze data generated specifically for recordings that included in-session Eye Gaze Calibration.
  6. Overview of the Aria Pilot Dataset

    The Aria Pilot Dataset is an open dataset of egocentric scenarios (e.g., cooking, exercising, gaming) captured using Project Aria devices. It includes raw sensor data and derived machine perception results such as per-frame eye tracking, 3D user trajectories, shared space-time information between multiple wearers, and speech-to-text annotations. The dataset consists of over 2.1 million images and 7.5 accumulated hours of data.
  7. Overview of Project Aria MPS CLI

    The Project Aria MPS Command Line Interface (MPS CLI) is the preferred tool for requesting Machine Perception Services (MPS). It allows users to upload VRS files to Meta servers for processing and automatically downloads the results to a local directory. It supports concurrent processing of multiple recordings.
  8. Preserve privacy using EgoBlur

    EgoBlur is an open-source AI model from Meta that detects and blurs Personally Identifiable Information (PII), specifically faces and license plates, using FasterRCNN-based detector models.

    Available Tools:

    • EgoBlur VRS Utilities: A C++ tool for processing Aria VRS files.
    • EgoBlur Demo: A Python3 tool for processing PNG, JPEG, or MP4 files.
  9. Understand the four quadrants of Emotional Intelligence

    Emotional Intelligence (EQ) can be categorized by how a person processes information (Knowing vs. Acting) and the target of that focus (Self vs. Others). This creates four distinct quadrants of ability:

    • Self-Awareness: Knowing your own emotions.
    • Self-Management: Acting on/managing your own emotions.
    • Social Awareness: Knowing the emotions and cues of others.
    • Relationship Management: Acting on that knowledge to manage interactions with others.
  10. Project Aria Hardware and Data Overview

    Project Aria glasses are multimodal data collection devices used for spatial AI and AR research. They capture data from a human perspective using the following sensors:

    • Five cameras: two Mono Scene, one RGB, and two Eye Tracking cameras.
    • Non-visual sensors: two IMUs, magnetometer, barometer, GPS, Wi-Fi beacon, Bluetooth beacon, and microphones.

    Data is stored in VRS files, which preserve the integrity of individual sensor streams and ensure they are time-aligned using shared timestamps.

  11. Apply Permaculture design principles to documentation

    Permaculture principles for sustainable land management can be adapted to documentation and architecture to ensure the process is sustainable and user-centric. Key applications include:

    • Listening to the ecosystem: Avoid forcing a specific structure onto documents without considering the specific workplace and user base.
    • Obtaining a yield: Focus on smaller, impactful user stories rather than getting lost in overly elaborate structures.
    • Using edges and the marginal: Accept that 'messy' is acceptable during the forming and storming phases of a project.
    • Energy storage: Prioritize big-picture architectural work over premature copy-editing perfection.
  12. Get started with Project Aria glasses

    For users who have received Project Aria glasses, the following resources are available for setup and operation:

    • Overview: Refer to the 'About ARK' documentation for device usage options.
    • Setup: Follow the 'Getting Started With Project Aria Glasses' quickstart guide.
    • Manual: Consult the 'Glasses User Manual' for detailed operations, including factory resets.
    • Support: Visit the official Support page for troubleshooting and contact options.