procedural-3d-ai-viz

name	procedural-3d-ai-viz
description	Workflow for building platforms that convert unstructured blueprints/URLs into interactive 3D visualizations using a JSON Scene Descriptor intermediate format.

Procedural 3D Scene Visualization with AI

This skill describes the workflow for building a platform that converts unstructured text/URLs (blueprints) into interactive 3D visualizations using a "JSON Scene Descriptor" intermediate format.

Core Concept

Instead of asking an LLM to generate complex 3D files (GLTF/OBJ) which are prone to syntax errors, the AI generates a standardized JSON schema describing primitives (shapes), their spatial coordinates, metadata, and logical connections. The frontend then renders these procedurally using Three.js.

Implementation Workflow

1. Schema Design (shared/schema.json)

Define a strict schema that includes:

• Parts: id, name, shape (box, cylinder, sphere), position [x, y, z], size [w, h, d], color, material, role, description.
• Connections: A mapping of IDs (e.g., part_a -> [part_b, part_c]).

2. AI Analyst Pipeline (apps/server/analyst.py)

• Prompt Engineering: Use a "PhD Mechanical Engineer" persona.
• Spatial Heuristics: Provide the AI with rules for plausible placement (e.g., "sensors should be on the periphery", "mounting plates should be at z=0").
• Format Enforcement: Mandate the output as a JSON object matching the schema.

3. Procedural Rendering (apps/web/src/components/Viewer.tsx)

• Use @react-three/fiber and @react-three/drei.
• Map the shape field from JSON to Three.js geometries (<boxGeometry />, <cylinderGeometry />).
• Implement a selection state to highlight parts and their associated metadata.

4. Connectivity Visualization (apps/web/src/components/Connections.tsx)

• Iterate through the connections array.
• Draw lines/cylinders between the center points of connected parts.
• Implement visual feedback (glowing lines) when a connected node is selected.

Pitfalls & Solutions

• Random Clusters: AI often places objects randomly. Solution: Define a "Chassis" or "Base" object in the prompt to anchor coordinates.
• Symmetry/Alignment: Primitives can overlap. Solution: Use a multi-step prompt (identify parts $\to$ define spatial layout $\to$ refine connections).
• CORS Issues: Frontend and Backend on different ports. Solution: Use fastapi.middleware.cors with allow_origins=["*"] (or specific domain) during development.