| name | augmented-reality-tracking |
| description | Tracking and registration fundamentals for augmented reality. Covers the six classes of tracking (marker, markerless, inertial, SLAM, outside-in, inside-out), registration error sources (calibration, latency, drift, distortion), the Azuma definition of AR, display configurations (optical see-through, video see-through, projection, handheld), and the interaction between tracking precision and user perception of "being there." Use when designing AR experiences, diagnosing registration issues, or selecting tracking technology. |
| type | skill |
| category | spatial-computing |
| status | stable |
| origin | tibsfox |
| modified | false |
| first_seen | "2026-04-12T00:00:00.000Z" |
| first_path | examples/skills/spatial-computing/augmented-reality-tracking/SKILL.md |
| superseded_by | null |
Augmented Reality Tracking
Augmented reality succeeds or fails on registration — the precise alignment of virtual content with the physical world. A perfectly rendered virtual object floating six inches from where it should be is worse than no augmentation at all. This skill catalogs the tracking techniques, error sources, and design heuristics that determine whether AR content feels anchored to reality or simply pasted over it.
Agent affinity: azuma (registration, tracking, AR definition), sutherland (first HMD), furness (head tracking and displays)
Concept IDs: spatial-coordinate-navigation, spatial-reasoning-3d, spatial-signal-propagation
The Azuma Definition of AR
Azuma (1997) defined AR as any system that:
- Combines real and virtual
- Is interactive in real time
- Is registered in 3D
This is the minimum bar. Systems that meet fewer than all three are not AR in the technical sense — a video feed with overlaid text is not AR (not 3D-registered); a prerendered overlay is not AR (not real-time); a VR headset is not AR (no real content). The definition scopes what this skill addresses.
Tracking Classes
| Class | Mechanism | Precision | Range | Cost | Failure modes |
|---|
| Marker-based | Recognized fiducial (ARTag, ARUCO, QR) | High | Line-of-sight to marker | Low | Occlusion, lighting |
| Markerless feature | Track natural features (corners, edges) | Medium-high | Feature-rich scenes | Medium | Textureless surfaces |
| Inertial (IMU) | Gyroscope + accelerometer | Medium (drifts) | Unlimited | Low | Drift over seconds |
| SLAM | Simultaneous localization and mapping | Medium-high | Unlimited | High | Dynamic scenes, low texture |
| Outside-in | External cameras watch trackers | Very high | Room-scale | High | Occlusion, range |
| Inside-out | Cameras on HMD watch environment | High | Unlimited | Medium | Feature loss in empty rooms |
Most modern AR uses a combination: visual-inertial SLAM combines IMU (high frequency, drifts slowly) with visual features (lower frequency, no drift) to get the best of both.
