Project Aria glasses, introduced by Meta (formerly Facebook), represent a significant leap in wearable devices. Announced as part of Meta's larger vision for the metaverse, these glasses are designed to capture rich contextual egocentric multi-modal data, providing a platform for developing augmented reality (AR) experiences grounded in real-world interactions. Aria glasses are built to collect data about the environment and user behavior while opening new frontiers for eye tracking, spatial navigation, and human-computer interaction research. NirdsLab recently became a research partner for the program researching contextualized advanced eye tracking. In this blog post, we will discuss the capabilities of Aria glasses and how we use them in eye-tracking research.
| Image may be NSFW. Clik here to view.  |
| Figure 1: Components of the glasses (https://facebookresearch.github.io) At the current stage of development at Meta, the Aria glasses take the form of a sensor array for future smart glasses, containing visual and non-visual sensors for multi-modal data collection. The sensory information captured by the glasses includes video, audio, locations, and motion from the wearer’s perspective. Table 1 summarizes the sensors available in Aria glasses. |
| Sensor | Description |
|---|---|
| Mono scene camera | Captures the world with a maximum peripheral vision with stereo overlap. Primarily used for the visual Simultaneous Localization and Mapping (SLAM) capabilities/functions. |
| RGB camera | Capture the point of view of the wearer in a higher resolution than the Mono Scene cameras. |
| Eye-Tracking camera | Two monochromatic eye-facing cameras for eye-tracking. |
| Inertial measurement unit (IMU) | Captures movements of the glasses through an accelerometer (linear) and a gyroscope (angular). |
| Magnetometer | Measures the ambient magnetic field. |
| Barometer and thermometer | Captures local air pressure and the thermometer. |
| Microphone | Microphone array comprising seven microphones, capable of capturing spatial audio at 48 kHz. |
| Global Navigation and Satellite System (GNSS) receiver | Supports global location capturing with GPS and Galileo constellations. |
| Wi-Fi and Bluetooth transceiver | Transceivers capable of regularly scanning and recording Received Signal Strength Indicator (RSSI) from Wi-Fi and Bluetooth beacons. |
| Image may be NSFW. Clik here to view.  |
| Figure 2: Data collection view with project aria glasses. (source: https://www.projectaria.com/datasets/aea/) |
Setting up Project Aria glasses
- Download and install the Desktop Companion App using the instructions available at https://facebookresearch.github.io/projectaria_tools/docs/ARK/ark_downloads#desktop.
- Log into the desktop companion app using the Project Aria credentials.
- Connect the Aria glasses to the desktop via USB.
- Once plugged in, the device will display as "Aria" and "active" under “My Device.”
- Select a profile to start recording (e.g., Profile0). See list of profiles: https://facebookresearch.github.io/projectaria_tools/docs/tech_spec/recording_profiles .
The Aria desktop dashboard provides the ability to record, process, transfer, and visualize data.
| Image may be NSFW. Clik here to view.  |
| Figure 3: Aria desktop application dashboard. (https://facebookresearch.github.io/projectaria_tools/docs/ARK/desktop_companion_app) |
Recording with the Mobile Companion App
- Download and install the mobile companion app compatible with the device from https://facebookresearch.github.io/projectaria_tools/docs/ARK/ark_downloads.
- Plug the Project Aria glasses into the charger (this will automatically switch on the glasses) or switch on the glasses using the button.
- Log into the mobile companion app (first time).
- Select “Get Started” to begin the setup.
- The app will search for the nearby Project Aria glasses, and once discovered, the glasses will appear in the app.
- Select the listed glasses to pair with the companion app.
- Once pairing is completed, the app will ask to name the Aria glasses.
- Join a Wi-Fi network; the glasses must be plugged into a charger to complete the setup process.
- Once connected to Wi-Fi, the glasses will look for updates and update the glasses' OS.
- Once setup is complete, the mobile companion app dashboard will appear.
- The glasses are now set to record. Unplug the glasses and start recording using a preferred profile.
The mobile app provides limited functionalities for interacting with Aria glasses, including setting up recording profiles, managing recordings, and monitoring device status.
| Image may be NSFW. Clik here to view.  |
| Figure 4: Aria mobile companion app dashboard. (https://facebookresearch.github.io/projectaria_tools/docs/ARK/mobile_companion_app) |
Aria open datasets for new research opportunities
The Project Aria research team provides open datasets collected with Project Aria glasses for new research. Aria’s Aria Everyday Activities (AEA) dataset contains anonymized Aria sequences captured in various scenarios, such as cooking, playing games, or exercising. This dataset is a valuable resource for analyzing first-person perspectives of daily activities.
TheAria Synthetic Environmentsdataset features interior layouts filled with 3D objects, simulated to match the sensor characteristics of Aria glasses. Unlike earlier 3D scene understanding datasets, which often lack sufficient scale for effective machine learning training, this dataset sets a new benchmark in size and scope. It opens up research opportunities in 3D scene reconstruction, object detection, and tracking.
Nymeria is the largest human motion dataset captured in natural settings, featuring diverse activities and locations. It is the first application to use multiple egocentric multimodal devices synchronized within a unified 3D metric framework. This dataset helps to advance research in egocentric human motion understanding, aiming to drive innovations in contextualized computing and future AR/VR technologies.
The HOT3D dataset addresses the challenge of understanding how humans use their hands to manipulate objects, a key area in computer vision research. This dataset opens new possibilities, such as transferring manual skills from experts to novices or robots, enabling AI assistants to interpret user actions.
The Digital Twin Catalog (DTC) dataset provides a highly detailed collection of sub-millimeter-level accurate 3D object models and source capture data for various reconstruction techniques. As reliance on 3D digital twins grows for e-commerce, entertainment, and advertising, DTC aims to reduce manual 3D model creation costs by enabling researchers to advance reconstruction techniques.
The Aria Everyday Objects (AEO) dataset provides a critical real-world validation tool for 3D object detection in egocentric data. It ensures that AI models trained on synthetic datasets like EFM3D and SceneScript can be generalized effectively to real-world scenarios beyond simulations.Eye tracking with Project Aria glasses
Project Aria glasses can estimate gaze direction from data acquired from eye-tracking cameras. For this purpose, they have developed the Aria Machine Perception Service (MPS) to estimate gaze information during post-processing steps through the MPS Command Line Interface (MPS CLI) or the desktop application. It is important to note that a recording profile with eye-tracking cameras needs to be enabled to access the eye-tracking information.
When eye tracking is combined with other spatial information, we can interpret the visual attention and environmental interactions of users in an environment. With Aria glasses, wireless wearable glasses eliminate device-level constraints such as movement restrictions due to wiring in multi-user eye-tracking with dedicated devices. In multi-user studies, project aria tools facilitate time-synchronized recordings with multiple project aria glasses. Further, in-built sensors allow us to capture additional events beyond visual attention, such as user movements and auditory events, making the glasses a robust platform for research in user behaviors and assistive technologies.
| Image may be NSFW. Clik here to view.  |
| Figure 5: Wearer’s point of view captured using the RGB camera during a user study in the NirdsLab. The green dot shows the gaze of the wearer. |
Research with Project Aria glasses in the NirdsLab
Accessibility research-Bhanuka (@mahanama94) Kumushini (@KumushiniT)