LensStudioAgents

Reference guide for 2D UI and screen-space interaction in Lens Studio — covering ScreenTransform anchors/offsets/size/pivot and coordinate conversions (localPointToScreenPoint, screenPointToLocalPoint, localPointToWorldPoint), ScreenImage (texture, stretch mode, color tint), Text component (content, font, alignment, color, size), ScreenRegionComponent for defining tap/touch areas, TouchComponent with TouchStartEvent/TouchMoveEvent/TouchEndEvent, tap input for phone lenses, LSTween UI animations (colorTo, moveTo on screen elements), multi-swatch color picker pattern, undo stack pattern, and common gotchas. Use this skill whenever a lens needs a 2D UI panel, tap interaction, on-screen buttons, text labels, color pickers, swipeable menus, or undo/redo — covering Drawing, Quiz, TappableQuestion, MemeSticker, MusicVideo, and HighScore examples.

Learn how to programmatically request and manage raw camera textures and crops.

Reference guide for face and body AR tracking in Lens Studio — covering FaceTrackingComponent setup (multi-face, faceIndex), FaceInset and FaceMask effects, 2D and 3D Face Attachments (hat/mouth/left_eye/right_eye anchors), Face Mesh UV texturing, Face Landmarks (68 keypoints), Face Expression weights for mouth-open/eye-blink detection, Eye Tracking component (left/right eye direction), Upper Body Tracking 3D asset (hips/spine/shoulder attachment points), Upper Body Mesh for seamless selfie occlusion, and Face Retouch/Eye Color/Face Liquify/Face Stretch effects. Use this skill for any phone or front-camera lens involving faces, selfie effects, makeup, face masks, 3D head ornaments, body tracking, or expression-driven animations — covering the vast majority of Snapchat lens content.

Learn how to dynamically instantiate prefabs algorithmically in grid or spherical patterns.

Implement proximity-based triggers and smooth tethering behaviors for interactive objects.

Learn how to track image markers and detach the tracked content into World Space in AR.

Reference guide for materials and shaders in Lens Studio — covering runtime material property changes (clone-before-modify, mainPass.baseColor, mainPass.opacity, mainPass.baseTex), blend modes (Normal/Alpha/Add/Screen/Multiply), depth and cull settings (depthTest, depthWrite, twoSided, cullMode), render order, material variants, assigning textures and render targets, reading and writing RenderTarget textures for post-processing, the graph-based Material Editor node system, custom shader graph nodes, and common shader pitfalls. Use this skill for any lens that needs to change material colors or textures at runtime, implement custom visual effects with shaders, set up post-processing render pipelines, chain render targets, or debug material/blend-mode issues — covering MaterialEditor, Drawing, and HairSimulation examples.

Reference guide for Lens Studio's math types and utilities — vec2/vec3/vec4/quat/mat4, MathUtils (DegToRad, RadToDeg, clamp, remap, lerp, inverseLerp), ScreenTransform 2D anchor/offset/size system, and Lens Studio's right-handed coordinate system (vec3.forward()=(0,0,-1)). Covers construction, arithmetic (uniformScale, dot, cross, normalize, distance), quaternion creation from Euler angles (pitch/yaw/roll XYZ order), quat.lookAt with parallel-vector guard, quat.slerp, combining rotations with multiply, getWorldTransform/setWorldTransform, mat4 inverse and transform extraction, and practical recipes: billboard, frame-rate-independent smooth follow, angle between directions, world-to-screen pixel position, project-to-plane, color lerp via vec4, ScreenTransform coordinate conversions, and parsing BLE sensor quaternion bytes. Use this skill for any 3D math, 2D UI positioning, position/rotation/scale arithmetic, or coordinate-space conversion in a Lens Studio TypeScript script.

Reference guide for the Lens Studio TypeScript component system — covering the @component, @input, @hint, @allowUndefined, and @label decorators, the BaseScriptComponent lifecycle (onAwake vs OnStartEvent, UpdateEvent, DelayedCallbackEvent one-shot and repeating timers, TurnOnEvent/TurnOffEvent, onDestroy), accessing components with getComponent (plus null-check patterns to fix 'cannot read property of null' errors), cross-TypeScript imports with getTypeName(), NativeLogger vs print, prefab instantiation (sync and async), SceneObject hierarchy queries, and enabling/disabling objects. Use this skill whenever writing or debugging any Lens Studio TypeScript script, wiring up scene objects, or fixing 'this is undefined' or null-reference errors — platform-agnostic (works for Spectacles and phone lenses).

Use TypeScript decorators to bind lifecycle events and inject dependencies declaratively.

Reference guide for Snapchat user data and social features in Lens Studio — covering UserContextSystem (display name, Bitmoji, profile picture), Bitmoji 2D (requestBitmoji2DResource + RemoteMediaModule fetch pattern), Bitmoji 3D (requestBitmoji3DResource, AnimationMixer for playback), Bitmoji Head with live facial animation, Friends API (FriendsComponent, FriendInfo, listing friends and their Bitmojis), Dynamic Response Poster/Responder mechanic (tappable areas, reading Poster data in the Responder flow with DynamicResponseComponent), and LeaderboardModule (Leaderboard Custom Component, Score Widget, submit/retrieve with OrderingType and UsersType). Use this skill whenever a lens needs the current user's name or avatar, accesses friends' Bitmoji or data, implements a send-and-respond mechanic, or adds a global score leaderboard.

Reference guide for Lens Studio's VFX Graph particle system — covering VFXComponent setup (VFX asset vs Component relationship), reading and writing .asset.properties to pass data into a VFX graph at runtime (position, color, float, direction vectors), particle system inspector settings (emitter rate, lifetime, shape, blending, world vs local simulation space), spawning particles from script (trigger emission via properties), sending scene object transforms and screen-space data to VFX, and common VFX bugs (type mismatches on properties, asset null checks). Use this skill whenever a lens needs particle effects, camera-reactive or gesture-driven VFX, attaching effects to face/body tracking anchors, or connecting real-time data (position, audio levels, expression weights) into a VFX graph.

Render smooth 3D tube geometry and splines dynamically for visual effects and drawing apps.

Reference guide for world understanding and scoring in Lens Studio — covering WorldQueryModule HitTestSession (HitTestSessionOptions.filter for jitter smoothing, semantic surface classification for floor/wall/ceiling/table detection, null result handling, per-frame performance), SIK InteractionManager targeting interactor ray pattern, Physics.createGlobalProbe().rayCast for scene-collider hits with collision layer filtering, aligning objects to surface normals using quat.lookAt, and the LeaderboardModule (create/retrieve with TTL and OrderingType, submitScore, getLeaderboardInfo with UsersType.Global/Friends). Use this skill when detecting real floors/walls/tables to place AR content, raycasting for hover or interaction against scene objects, or adding a global in-lens leaderboard — differentiates from spectacles-lens-essentials (physics/SIK) and from spectacles-cloud (Supabase persistence).

Reference guide for AI-powered AR lenses using the Remote Service Gateway (RSG) — covering LLM calls (OpenAI, Gemini, Claude), AI music generation (Lyria), ASR speech-to-text with language/accuracy options, TTS text-to-speech, camera frame capture and base64 encoding for vision models, streaming token-by-token responses via onPartialResponse, multi-turn conversation history, and agentic tool-call loops. Use this skill whenever a lens needs to call an LLM or AI model via RSG, stream AI tokens to a UI text element, transcribe voice (ASR) or synthesise speech (TTS), send a camera image to a cloud vision API, or run a multi-step autonomous loop. Do NOT use this for plain REST calls to non-AI backends — use spectacles-networking for those.

Reference guide for implementing OAuth2 Authentication in Spectacles Lenses (AuthKit) — covering Authorization Code Flow with PKCE, Implicit Flow, DeepLinkModule for redirect callbacks, InternetModule for token swaps, TokenManager with GeneralDataStore, handling state verification, and access token refresh. Use this skill when a lens requires user authentication with third-party APIs (Spotify, GitHub, custom backends) using OAuth2.

Reference guide for Bluetooth Low Energy (BLE) in Spectacles lenses — covering enabling the experimental BleModule API (Project Settings → Spectacles → Experimental APIs → Bluetooth), scanning for peripherals by service UUID (with stopScan for battery), connecting, discoverServices/discoverCharacteristics, read-once and notify-stream characteristic patterns, DataView byte parsing and little-endian endianness, writing command bytes and floats, MTU negotiation with requestMtu, the disconnection/reconnect loop, BLE MTU limits (20 bytes default, chunking for larger payloads), and the Spectacles Mobile Kit for phone↔lens JSON messaging. Use this skill for any lens that connects to Arduino hardware, a game controller, a BLE sensor, or a companion mobile app — covering BLE Arduino, BLE Playground, BLE Game Controller, and Spectacles Mobile Kit samples.

Reference guide for Snap Cloud (Supabase-powered backend) in Spectacles lenses — covering Fetch API setup (requires Internet Access capability in Project Settings), Postgres REST queries with the anon key, Row Level Security policies, Realtime WebSocket subscriptions with correct postgres_changes event format and reconnect-on-sleep patterns, cloud storage uploads of base64 images captured by Spectacles, serverless Edge Functions, and companion web dashboard architecture. Use this skill whenever a lens needs persistent cloud data, needs to share data with a web app in real time, uploads captured images to a bucket, or calls a cloud function — covering Snap Cloud and World Kindness Day samples. Use spectacles-networking for plain REST calls to non-Snap backends, and spectacles-connected-lenses for in-session multiplayer state.

Reference guide for CommerceKit and In-App Purchases on Spectacles — covering CommerceKitModule initialization, converting prices to integer cents, editor mock purchases versus device production purchases, queryPurchaseHistory for ownership checking, launchPurchaseFlow to prompt a swipe-to-buy, acknowledgePurchase to verify consumption, and handling UserCanceled responses. Use this skill when building premium lenses, freemium features, unlocking levels, or selling non-consumable digital items on the Snap platform.

Reference guide for real-time multiplayer AR on Spectacles using Connected Lenses and Spectacles Sync Kit — covering session creation/joining with joinOrCreateSession (including 'already-in-session' error handling), TransformSyncComponent for position/rotation replication, RealtimeStore for shared key-value state (max 512 bytes per key), NetworkEventSystem for one-shot broadcast events, EntityOwnership for physics authority, Lens Cloud for persistent cross-session data, and patterns for turn-based (Tic Tac Toe) and real-time physics (Air Hockey). Also covers late-joiner state sync, transform drift mitigation, and store size limits. Use this skill whenever multiple Spectacles users need to share AR objects or state — covering Tic Tac Toe, Air Hockey, Laser Pointer, High Five, Shared Sync Controls, Spectacles Sync Kit, and Think Out Loud samples.

Reference guide for foundational Lens Studio patterns on Spectacles — covering the GestureModule (pinch down/up/strength, targeting, grab, phone-in-hand with correct TypeScript API), SIK components (PinchButton, DragInteractable, GrabInteractable, ScrollView), hand-tracking gestures, physics bodies/colliders/callbacks (including audio-on-collision), LSTween animation (position/scale/rotation/color tweens), prefab instantiation at runtime, materials (clone-before-modify), spatial anchors, on-device persistent storage (putString/getFloat), spatial images, and the Path Pioneer raycasting pattern. Use this skill for any Spectacles lens that needs interaction, motion, animation, physics, audio, or persistent local storage — including Essentials, Throw Lab, Spatial Persistence, Spatial Image Gallery, Path Pioneer, Public Speaker, Voice Playback, Material Library, and DJ Specs samples.

Connect Sony Mocopi motion capture hardware to Spectacles using WebSockets for real-time skeletal tracking.

Reference guide for location-based AR and navigation on Snap Spectacles — covering Custom Locations scanning and localisation events (including full LocalisationStatus enum: Localised, NotLocalised, LocalisationLost, Unavailable, plus failure/rescan patterns), the Mapbox Map Component (zoom, tile loading, mapStyle), the Snap Places API (POI search, routing), outdoor turn-by-turn navigation with a compass-bearing AR arrow, indoor waypoint navigation with Dijkstra, the GeoLocation API for live GPS (requires Location capability), and coordinate conversions between GPS and world space. Use this skill for any lens that anchors AR to a real-world room, building, or outdoor place, shows a map, guides the user somewhere, or queries nearby businesses — covering Custom Locations, Outdoor Navigation, and Navigation Kit samples.

Reference guide for the Lens Studio Fetch API and WebView component in Spectacles lenses — covering InternetModule (Lens Studio 5.9+), Fetch API via internetModule.fetch(Request) with bytes/text/json response handling, performHttpRequest, Internet Access capability, GET/POST requests, custom headers, Bearer auth, polling, timeouts, CORS/HTTPS, WebSocket and RemoteMediaModule for media from URLs, and bidirectional WebView messaging. Use this skill for any lens that calls a REST API, polls a JSON endpoint, loads remote images, embeds a webpage, or talks to a custom backend — including the Fetch sample. Use spectacles-ai for LLM/RSG calls, or spectacles-cloud for Supabase/Snap Cloud integration.

Reference guide for on-device machine learning in Spectacles lenses using SnapML — covering MLComponent inspector configuration, supported model formats (TFLite/ONNX), input tensor shape setup, synchronous (runImmediate) and asynchronous (runScheduled) inference patterns, parsing SSD/YOLO bounding box output tensors, INT8 quantization for NPU performance (avoid FP32 layers), low-pass filter smoothing to reduce jitter, ObjectTracking3D for persistent tracking, and integrating detections with physics colliders. Use this skill whenever a lens needs to run a custom ML model on Spectacles without a cloud call, or when debugging input-shape mismatches — including SnapML Starter, SnapML Chess Hints, SnapML Pool. Use spectacles-ai instead if the model runs in the cloud via RSG.

Learn how to persist and restore AR content across sessions using World Anchors.

Reference guide for Spectacles UI Kit (FastUI/UIKit) — covering programmatic UI generation without prefabs. Includes Frame configuration (Large/Small appearance, autoShowHide, setFollowing), GridLayout (rows, columns, cellSize vec2, cellPadding vec4), and programmatic Button creation (RectangleButton, CapsuleButton, RoundButton). Shows the correct creation lifecycle (createComponent → set size → initialize() → set properties → bind events). Use this skill when building complex 2D spatial layouts, grids, menus, dialogs, or galleries entirely in TypeScript instead of assembling them manually in the Lens Studio inspector.