axiom-realitykit-diag

name	axiom-realitykit-diag
description	Use when RealityKit entities not visible, anchors not tracking, gestures not responding, performance drops, materials wrong, or multiplayer sync fails
license	MIT
metadata	{"version":"1.0.0"}

RealityKit Diagnostics

Systematic diagnosis for common RealityKit issues with time-cost annotations.

When to Use This Diagnostic Skill

Use this skill when:

• Entity added but not visible in the scene
• AR anchor not tracking or content floating
• Tap/drag gestures not responding on 3D entities
• Frame rate dropping or stuttering
• Material looks wrong (too dark, too bright, incorrect colors)
• Multiplayer entities not syncing across devices
• Physics bodies not colliding or passing through each other

For RealityKit architecture patterns and best practices, see axiom-realitykit. For API reference, see axiom-realitykit-ref.

---

Mandatory First Step: Enable Debug Visualization

Time cost: 10 seconds vs hours of blind debugging

// In your RealityView or ARView setup
#if DEBUG
// Xcode: Debug → Attach to Process → Show RealityKit Statistics
// Or enable in code:
arView.debugOptions = [
    .showStatistics,       // Entity count, draw calls, FPS
    .showPhysics,          // Collision shapes
    .showAnchorOrigins,    // Anchor positions
    .showAnchorGeometry    // Detected plane geometry
]
#endif

If you can't see collision shapes with .showPhysics, your CollisionComponent is missing or misconfigured. Fix collision before debugging gestures or physics.

---

Symptom 1: Entity Not Visible

Time saved: 30-60 min → 2-5 min

Entity added but nothing appears
│
├─ Is the entity added to the scene?
│   └─ NO → Add to RealityView content:
│        content.add(entity)
│        ✓ Entities must be in the scene graph to render
│
├─ Does the entity have a ModelComponent?
│   └─ NO → Add mesh and material:
│        entity.components[ModelComponent.self] = ModelComponent(
│            mesh: .generateBox(size: 0.1),
│            materials: [SimpleMaterial(color: .red, isMetallic: false)]
│        )
│        ✓ Bare Entity is invisible — it's just a container
│
├─ Is the entity's scale zero or nearly zero?
│   └─ CHECK → Print: entity.scale
│        USD models may import with unexpected scale.
│        Try: entity.scale = SIMD3(repeating: 0.01) for meter-scale models
│
├─ Is the entity behind the camera?
│   └─ CHECK → Print: entity.position(relativeTo: nil)
│        In RealityKit, -Z is forward (toward screen).
│        Try: entity.position = SIMD3(0, 0, -0.5) (half meter in front)
│
├─ Is the entity inside another object?
│   └─ CHECK → Move to a known visible position:
│        entity.position = SIMD3(0, 0, -1)
│
├─ Is the entity's isEnabled set to false?
│   └─ CHECK → entity.isEnabled = true
│        Also check parent: entity.isEnabledInHierarchy
│
├─ Is the entity on an untracked anchor?
│   └─ CHECK → Verify anchor is tracking:
│        entity.isAnchored (should be true)
│        If using plane anchor, ensure surface is detected first
│
└─ Is the material transparent or OcclusionMaterial?
    └─ CHECK → Inspect material:
         If using PhysicallyBasedMaterial, check baseColor is not black
         If using blending = .transparent, check opacity > 0

Quick Diagnostic

func diagnoseVisibility(_ entity: Entity) {
    print("Name: \(entity.name)")
    print("Is enabled: \(entity.isEnabled)")
    print("In hierarchy: \(entity.isEnabledInHierarchy)")
    print("Is anchored: \(entity.isAnchored)")
    print("Position (world): \(entity.position(relativeTo: nil))")
    print("Scale: \(entity.scale)")
    print("Has model: \(entity.components[ModelComponent.self] != nil)")
    print("Children: \(entity.children.count)")
}

---

Symptom 2: Anchor Not Tracking

Time saved: 20-45 min → 3-5 min

AR content not appearing or floating
│
├─ Is the AR session running?
│   └─ For RealityView on iOS 18+, AR runs automatically
│       For ARView, check: arView.session.isRunning
│
├─ Is SpatialTrackingSession configured? (iOS 18+)
│   └─ CHECK → Ensure tracking modes requested:
│        let config = SpatialTrackingSession.Configuration(
│            tracking: [.plane, .object])
│        let result = await session.run(config)
│        if let notSupported = result {
│            // Handle unsupported modes
│        }
│
├─ Is the anchor type appropriate for the environment?
│   ├─ .plane(.horizontal) → Need a flat surface visible to camera
│   ├─ .plane(.vertical) → Need a wall visible to camera
│   ├─ .image → Image must be in "AR Resources" asset catalog
│   ├─ .face → Front camera required (not rear)
│   └─ .body → Full body must be visible
│
├─ Is minimumBounds too large?
│   └─ CHECK → Reduce minimum bounds:
│        AnchorEntity(.plane(.horizontal, classification: .any,
│            minimumBounds: SIMD2(0.1, 0.1)))  // Smaller = detects sooner
│
├─ Is the device supported?
│   └─ CHECK → Plane detection requires A12+ chip
│       Face tracking requires TrueDepth camera
│       Body tracking requires A12+ chip
│
└─ Is the environment adequate?
    └─ CHECK → AR needs:
         - Adequate lighting (not too dark)
         - Textured surfaces (not blank walls)
         - Stable device position during initial detection

---

Symptom 3: Gesture Not Responding

Time saved: 15-30 min → 2-3 min

Tap/drag on entity does nothing
│
├─ Does the entity have a CollisionComponent?
│   └─ NO → Add collision shapes:
│        entity.generateCollisionShapes(recursive: true)
│        // or manual:
│        entity.components[CollisionComponent.self] = CollisionComponent(
│            shapes: [.generateBox(size: SIMD3(0.1, 0.1, 0.1))])
│        ✓ Collision shapes are REQUIRED for gesture hit testing
│
├─ [visionOS] Does the entity have InputTargetComponent?
│   └─ NO → Add it:
│        entity.components[InputTargetComponent.self] = InputTargetComponent()
│        ✓ Required on visionOS for gesture input
│
├─ Is the gesture attached to the RealityView?
│   └─ CHECK → Gesture must be on the view, not the entity:
│        RealityView { content in ... }
│            .gesture(TapGesture().targetedToAnyEntity().onEnded { ... })
│
├─ Is the collision shape large enough to hit?
│   └─ CHECK → Enable .showPhysics to see shapes
│        Shapes too small = hard to tap.
│        Try: .generateBox(size: SIMD3(repeating: 0.1)) minimum
│
├─ Is the entity behind another entity?
│   └─ CHECK → Front entities may block gestures on back entities
│        Ensure collision is on the intended target
│
└─ Is the entity enabled?
    └─ CHECK → entity.isEnabled must be true
         Disabled entities don't receive input

Quick Diagnostic

func diagnoseGesture(_ entity: Entity) {
    print("Has collision: \(entity.components[CollisionComponent.self] != nil)")
    print("Has input target: \(entity.components[InputTargetComponent.self] != nil)")
    print("Is enabled: \(entity.isEnabled)")
    print("Is anchored: \(entity.isAnchored)")

    if let collision = entity.components[CollisionComponent.self] {
        print("Collision shapes: \(collision.shapes.count)")
    }
}

---

Symptom 4: Performance Problems

Time saved: 1-3 hours → 10-20 min

Frame rate dropping or stuttering
│
├─ How many entities are in the scene?
│   └─ CHECK → Print entity count:
│        var count = 0
│        func countEntities(_ entity: Entity) {
│            count += 1
│            for child in entity.children { countEntities(child) }
│        }
│        Under 100: unlikely to be entity count
│        100-500: review for optimization
│        500+: definitely needs optimization
│
├─ Are mesh/material resources shared?
│   └─ NO → Share resources across identical entities:
│        let sharedMesh = MeshResource.generateBox(size: 0.05)
│        let sharedMaterial = SimpleMaterial(color: .white, isMetallic: false)
│        // Reuse for all instances
│        ✓ RealityKit batches entities with identical resources
│
├─ Is a System creating components every frame?
│   └─ CHECK → Look for allocations in update():
│        Creating ModelComponent, CollisionComponent, or materials
│        every frame causes GC pressure.
│        Cache resources, only update when values change.
│
├─ Are collision shapes mesh-based?
│   └─ CHECK → Replace generateCollisionShapes(recursive: true)
│        with simple shapes (box, sphere, capsule) for dynamic entities
│
├─ Is generateCollisionShapes called repeatedly?
│   └─ CHECK → Call once during setup, not every frame
│
├─ Are there too many physics bodies?
│   └─ CHECK → Dynamic bodies are most expensive.
│        Convert distant/static objects to .static mode.
│        Remove physics from non-interactive entities.
│
└─ Is the model polygon count too high?
    └─ CHECK → Decimate models for real-time use.
         Target: <100K triangles total for mobile AR.
         Use LOD (Level of Detail) for distant objects.

---

Symptom 5: Material Looks Wrong

Time saved: 15-45 min → 5-10 min

Colors, lighting, or textures look incorrect
│
├─ Is the scene too dark?
│   └─ CHECK → Missing environment lighting:
│        Add DirectionalLightComponent or EnvironmentResource
│        In AR, RealityKit uses real-world lighting automatically
│        In non-AR, you must provide lighting explicitly
│
├─ Is the baseColor set?
│   └─ CHECK → PhysicallyBasedMaterial defaults to white
│        material.baseColor = .init(tint: .red)
│        If using a texture, verify it loaded:
│        try TextureResource(named: "albedo")
│
├─ Is metallic set incorrectly?
│   └─ CHECK → metallic = 1.0 makes surfaces mirror-like
│        Most real objects: metallic = 0.0
│        Only metals (gold, silver, chrome): metallic = 1.0
│
├─ Is the texture semantic wrong?
│   └─ CHECK → Use correct semantic:
│        .color for albedo/baseColor textures
│        .raw for data textures (metallic, roughness)
│        .normal for normal maps
│        .hdrColor for HDR textures
│
├─ Is the model upside down or inside out?
│   └─ CHECK → Try:
│        material.faceCulling = .none (shows both sides)
│        If that fixes it, the model normals are flipped
│
└─ Is blending/transparency unexpected?
    └─ CHECK → material.blending
         Default is .opaque
         For transparency: .transparent(opacity: ...)

---

Symptom 6: Physics Not Working

Time saved: 20-40 min → 5-10 min

Objects pass through each other or don't collide
│
├─ Do both entities have CollisionComponent?
│   └─ NO → Both sides of a collision need CollisionComponent
│
├─ Does the moving entity have PhysicsBodyComponent?
│   └─ NO → Add physics body:
│        entity.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
│            mode: .dynamic)
│
├─ Are collision groups/filters configured correctly?
│   └─ CHECK → Entities must be in compatible groups:
│        Default: group = .default, mask = .all
│        If using custom groups, verify mask includes the other group
│
├─ Is the physics mode correct?
│   ├─ Two .static bodies → Never collide (both immovable)
│   ├─ .dynamic + .static → Correct (common setup)
│   ├─ .dynamic + .dynamic → Both move on collision
│   └─ .kinematic + .dynamic → Kinematic pushes dynamic
│
├─ Is the collision shape appropriate?
│   └─ CHECK → .showPhysics debug option
│        Shape may be too small, offset, or wrong type
│
└─ Are entities on different anchors?
    └─ CHECK → "Physics bodies and colliders affect only
         entities that share the same anchor" (Apple docs)
         Move entities under the same anchor for physics interaction

---

Symptom 7: Multiplayer Sync Issues

Time saved: 30-60 min → 10-15 min

Entities not appearing on other devices
│
├─ Does the entity have SynchronizationComponent?
│   └─ NO → Add it:
│        entity.components[SynchronizationComponent.self] =
│            SynchronizationComponent()
│
├─ Is the MultipeerConnectivityService set up?
│   └─ CHECK → Verify MCSession is connected before syncing
│
├─ Are custom components Codable?
│   └─ NO → Non-Codable components don't sync
│        struct MyComponent: Component, Codable { ... }
│
├─ Does the entity have an owner?
│   └─ CHECK → Only the owner can modify synced properties
│        Request ownership before modifying:
│        entity.requestOwnership { result in ... }
│
└─ Is the entity anchored?
    └─ CHECK → Unanchored entities may not sync position correctly
         Use a shared world anchor for reliable positioning

---

Common Mistakes

Mistake	Time Cost	Fix
No CollisionComponent on interactive entity	15-30 min	entity.generateCollisionShapes(recursive: true)
Missing InputTargetComponent on visionOS	10-20 min	Add InputTargetComponent()
Gesture on wrong view (not RealityView)	10-15 min	Attach .gesture() to RealityView
Entity scale wrong for USD model	15-30 min	Check units: meters vs centimeters
No lighting in non-AR scene	10-20 min	Add DirectionalLightComponent
Storing entity refs in System	30-60 min crash debugging	Query with EntityQuery each frame
Components not registered	10-15 min	Call registerComponent() in app init
Systems not registered	10-15 min	Call registerSystem() before scene load
Physics across different anchors	20-40 min	Put interacting entities under same anchor
Calling generateCollisionShapes every frame	Performance degradation	Call once during setup

---

Diagnostic Quick Reference

Symptom	First Check	Time Saved
Not visible	Has ModelComponent? Scale > 0?	30-60 min
No gesture response	Has CollisionComponent?	15-30 min
Not tracking	Anchor type matches environment?	20-45 min
Frame drops	Entity count? Resource sharing?	1-3 hours
Wrong colors	Has lighting? Metallic value?	15-45 min
No collision	Both have CollisionComponent? Same anchor?	20-40 min
No sync	SynchronizationComponent? Codable?	30-60 min
Sim OK, device crash	Metal features? Texture format?	15-30 min

---

Symptom 8: Works in Simulator, Crashes on Device

Time cost: 15-30 min (often misdiagnosed as model issue)

Q1: Is the crash a Metal error (MTLCommandBuffer, shader compilation)?
├─ YES → Simulator uses software rendering, device uses real GPU
│   Common causes:
│   - Custom Metal shaders with unsupported features
│   - Texture formats not supported on device GPU
│   - Exceeding device texture size limits (max 8192x8192 on older)
│   Fix: Check device GPU family, use supported formats
│
└─ NO → Check next

Q2: Is it an out-of-memory crash?
├─ YES → Simulator has more RAM available
│   Common: Large USDZ files with uncompressed textures
│   Fix: Compress textures, reduce polygon count, use LOD
│   Check: USDZ file size (keep < 50MB for reliable loading)
│
└─ NO → Check next

Q3: Is it an AR-related crash (camera, tracking)?
├─ YES → Simulator has no real camera/sensors
│   Fix: Test AR features on device only, use simulator for UI/layout
│
└─ NO → Check device capabilities
    - A12+ required for RealityKit
    - LiDAR for scene reconstruction
    - TrueDepth for face tracking

---

Resources

WWDC: 2019-603, 2019-605, 2023-10080, 2024-10103

Docs: /realitykit, /realitykit/entity, /realitykit/collisioncomponent, /realitykit/physicsbodycomponent

Skills: axiom-realitykit, axiom-realitykit-ref