| name | 3d-interaction-design |
| description | Design heuristics for three-dimensional interaction — input modalities (6DoF controllers, gaze, gesture, voice), selection and manipulation techniques (ray-cast, virtual hand, go-go, world-in-miniature), navigation (teleport, continuous, redirected walking), and feedback loops (haptic, audio, visual). Covers Fitts' law in 3D, discoverability vs affordance, safe-zone design, and comfort/simulator sickness mitigation. Use when designing VR/AR interactions, immersive walkthroughs, or any input system where the user's body is the controller. |
| type | skill |
| category | spatial-computing |
| status | stable |
| origin | tibsfox |
| modified | false |
| first_seen | "2026-04-12T00:00:00.000Z" |
| first_path | examples/skills/spatial-computing/3d-interaction-design/SKILL.md |
| superseded_by | null |
3D Interaction Design
Interaction in three dimensions is fundamentally different from interaction in two. Every designer trained on windows and cursors must unlearn assumptions about planar layout, pixel-precise pointing, and implicit desktop context. This skill catalogs the major input modalities, selection and manipulation techniques, navigation metaphors, and feedback strategies, with heuristics for when each applies.
Agent affinity: bret-victor (direct manipulation and feedback), sutherland (input device foundations), krueger (gestural and responsive interaction)
Concept IDs: spatial-reasoning-3d, spatial-coordinate-navigation, spatial-iterative-build-process
Input Modalities
| Modality | Degrees of freedom | Precision | Fatigue | Best for |
|---|
| 6DoF controller | 6 per hand | High | Low | VR manipulation, precise placement |
| Gaze | 2 (direction) | Medium | Low | Targeting, dwell-select |
| Gesture (free-hand) | Variable | Medium-low | High | Expressive, social, short sessions |
| Voice | Symbolic | High (recognition) | None | Labeling, commands, dictation |
| Eye tracking | 2 | High | None | Foveated rendering, implicit targeting |
| Brain-computer | Symbolic | Very low | Variable | Research, accessibility |
| Props / tangibles | Varies | High | Low | Authoring, haptic grounding |
The designer's first decision is which modality or combination the interaction will use. Voice plus gaze is a common accessibility-friendly pair. 6DoF plus audio is a common VR game pair. Gesture plus projected video is Krueger's VIDEOPLACE legacy.
Selection Techniques
Selecting an object in 3D is the precursor to manipulating it. The difficulty depends on distance, occlusion, density of candidates, and input modality.
Ray-cast (pointing)
Cast a ray from the controller (or from the head) into the scene. The first intersected object is the candidate. Simple, familiar, works at arbitrary distance. Fails when candidates are densely packed or partially occluded.
Virtual hand
The user's hand (represented by a 3D model) reaches into the scene and touches the object. Direct, tangible, satisfying. Limited by arm length.
