projectaria-plugins

Encoded knowledge for Project Aria Gen 2 and the Aria Research Kit (ARK) — hardware, profiles, calibration, coordinate systems, VRS format, time domains, on-device perception, MPS outputs (SLAM, point cloud, hand tracking), plus pointers to the source-of-truth docs for specifics. Use whenever the user asks ANY conceptual question about Aria Gen 2 / ARK and you need to either answer from encoded knowledge or know where to look next. This skill is the single index that tells an agent how to fetch official Aria knowledge — encoded stable concepts for what does not drift, explicit public-doc pointers for hardware specs / schemas / numerical values that do, and "go read the code" or "run --help" for things owned by source. Pair with the domain skills in this plugin (`projectaria-tools`, `client-sdk`, `mps`, `vrs-health-check`, `pilot-dataset`, `web-app-creator`, `custom-profile`, `vrs-cli`, `client-sdk-ros2-integration`, `cloud-streaming`) for hands-on work.

Use when integrating Aria Gen 2 sensor streams with ROS2 — publishing raw Aria sensor data through ROS2 topics, decoding it in subscriber nodes, sharing device calibration across nodes. The integration uses a custom `AriaRaw` message (a thin FlatBuffer wrapper) and a required `aria_data_types` package name. Use whenever the user asks about Aria + ROS2, publishing Aria data to topics, or building ROS2 nodes around Aria sensors.

Use when working with the Aria Client SDK — device pairing, recording, streaming, CLI commands (aria_gen2), Python SDK (aria.sdk_gen2), streaming visualization, multi-device time domain mapping, and device management. Use whenever the user mentions aria_gen2 CLI, aria.sdk_gen2, device pairing, recording start/stop, streaming from Aria glasses, or asks about controlling Aria Gen2 devices programmatically.

Use when configuring Aria Gen 2 glasses to stream sensor data directly to an internet-accessible HTTPS endpoint — instead of the default local USB / same-network WiFi streaming to a nearby PC. For field deployments, centralized multi-device collection, and cloud processing pipelines. Use whenever the user asks about remote streaming, streaming over the internet, cloud collection servers, or streaming without a laptop on hand.

Use when working with Aria Machine Perception Services (MPS) — cloud-based processing of Aria VRS recordings for SLAM trajectories, hand tracking, semi-dense point clouds, and online calibration. Covers the aria_mps CLI, single and multi-sequence processing, output formats, data lifecycle, and loading MPS results with ProjectAriaTools. Use whenever the user mentions MPS, aria_mps, SLAM processing, cloud hand tracking, trajectory generation, or submitting VRS files for processing.

Use when working with projectaria_tools (PAT) — the official Python/C++ library for reading Aria VRS recordings, accessing sensor data, device calibration, loading MPS results (eye gaze, hand tracking, SLAM trajectory), time domain mapping for multi-device alignment, and data visualization. Use whenever the user imports projectaria_tools, works with Aria VRS files, or asks about Aria sensor data processing.

Use when authoring a custom profile JSON for Aria Gen 2. A profile fixes which sensors are enabled and at what rate, resolution, and encoding — **the same profile works for both recording and streaming**, despite the historical "recording profile" name. Custom profiles carry real risks (thermal shutdown, fragmented data, MPS rejection), so this skill encodes the conservative authoring workflow plus the full released profile schema (top-level keys, per-sensor fields, enums, validation rules, rate-handling behaviors). Use whenever the user asks how to write a custom profile, modify a profile, define sensor rates, deploy a profile to the device, or asks what a specific profile field means.

Use when the user wants to inspect, validate, or manipulate a VRS file using the native VRS command-line tools from facebookresearch/vrs — `vrs check`, `vrs print`, etc. VRS is the file format used by Aria recordings, and the native CLI is separate from `projectaria-tools` (PAT). Reach for it for file-level operations (validate, list streams, slice, copy) when you do not need Aria sensor semantics. Use whenever the user asks about the `vrs` CLI, validating a VRS file outside PAT, or quick VRS inspection without writing Python.

Build webapps that integrate with Meta Aria Gen2 (Nebula) smart glasses, usable by any agent that supports skills (Manus, Claude Code, etc.). macOS only — the backend launcher and auto-`open` browser hooks are written for macOS; Windows and Linux are not supported. Covers the full Web App Creator stack: real-time sensor data via WebSocket at ws://localhost:17300 (VIO, hand tracking, eye gaze, IMU, PPG, audio, camera), the nebulaDataUpdate event bridge pattern used by Web App Creator, and the complete bidirectional voice pipeline (VAD, Whisper STT, hallucination filtering, LLM chat, TTS echo suppression). Use this skill for any Aria-connected webapp — from simple data visualizations to full conversational AI experiences.

Use when working with the Aria Gen2 Pilot Dataset — a multi-participant egocentric dataset with raw VRS sensor data, MPS outputs, and additional algorithm results (ASR, heart rate, hand-object interaction, depth estimation, 3D detection). Covers the projectaria-gen2-pilot-dataset package, data loaders, visualizers, and tutorials. Use whenever the user mentions pilot dataset, Gen2 pilot dataset, loading pilot data, pilot data loader, pilot dataset visualizer, projectaria_gen2_pilot_dataset, or any task involving the Aria Gen2 Pilot Dataset recordings or algorithm outputs.

Use when working with VRS Health Check — validating Aria recording quality before MPS processing. Covers the run_vrs_health_check CLI, configurations, threshold checks, exit codes, overriding thresholds, and the Python API. Use whenever the user mentions VRS health check, recording validation, data quality checks, dropped frames, or pre-MPS validation.