gazefix-obs-filter-dev

name	gazefix-obs-filter-dev
description	Guidance for developing the Gazefix OBS Studio macOS video filter plugin (libobs) and Streamlabs support paths. Use when implementing OBS sources/filters, frame pipelines, Vision tracking off-thread, GPU warps, plugin packaging/signing, or documenting Streamlabs Desktop compatibility.

gazefix-obs-filter-dev

When to use

• Building an OBS Studio plugin (sources/filters) for macOS
• Implementing the Gazefix OBS video filter pipeline (frame ingestion, backpressure, threading)
• Integrating Vision tracking in an OBS context (off render thread, bounded work)
• Implementing GPU warp rendering for gaze correction (Metal-backed on macOS)
• Adding debug overlays/metrics for OBS validation
• Packaging/distribution for macOS (.plugin bundle, installers, signing/notarization)
• Any user request mentioning OBS, Streamlabs, filters, libobs, or “plugin”

Core constraints (non-negotiables)

• Never block OBS render/video callbacks. Treat them like real-time paths.
• Bound latency: latest-frame-wins (overwrite slots / drop frames) instead of queueing.
• Fail-safe: if tracking confidence is low or stale, output pass-through frames.
• Local-first: no per-frame networking; any licensing/activation must not touch the hot path.
• Minimize copies: avoid full-frame CPU readbacks; downscale for Vision; keep warp on GPU.
• Apple Silicon only: the OBS filter is currently arm64-only. Avoid x86_64/universal build instructions unless Intel support is explicitly reintroduced.
• ALWAYS update documentation when behavior changes (docs, build/packaging steps, troubleshooting, and agent guidance).

Working model (recommended architecture)

• Filter instance per source:
  • Render path reads an atomic snapshot of latest warp params/confidence.
  • Worker path produces those params asynchronously.
• Backpressure:
  • One latest-frame slot (or tiny ring) feeding the worker.
  • One latest-params slot feeding the renderer.
  • Under load: drop intermediate frames; keep output responsive.
• Tracking cadence:
  • Run Vision at a capped rate (e.g. 15–30Hz), independent from render FPS.
  • Prefer downscaled/normalized inputs to keep time budget stable.

Implementation checklist

1. Locate the module boundary
   • OBS integration should live outside packages/GazefixCore/ and extension/ (e.g. obs/).
2. Decide reuse strategy
   • Prefer reusing packages/GazefixCore/ logic via a stable boundary (often a C ABI wrapper) to avoid divergence.
3. Design threading + data flow
   • Render thread: read-only; zero waits.
   • Worker: does conversion + Vision + smoothing; publishes params; never grows queues.
4. Implement confidence gating
   • Define “good/ok/bad” confidence bands and add hysteresis to avoid flicker.
5. GPU warp
   • Implement ROI-limited warp with soft mask blending.
   • Use ellipse fitting from eyelid contours for better mask shape (Issue #114).
   • Apply temporal stabilization to mask parameters to reduce flicker.
   • Clamp displacement aggressively near boundaries to reduce artifacts.
   • Preallocate buffers; update uniforms only.
6. Instrumentation
   • Track cadence, ms per track, drops, stale time, confidence.
   • Keep overlay optional and cheap.
7. Packaging
   • macOS plugin bundles live under ~/Library/Application Support/obs-studio/plugins/.
   • For distribution: sign + notarize; document install/uninstall.
8. Streamlabs support
   • Default recommendation: OBS Studio + Streamlabs Plugin for OBS.
   • Streamlabs Desktop compatibility is best-effort; provide the virtual camera fallback.
9. Documentation updates
   • Update the most relevant docs/*.md pages (install/setup, troubleshooting, performance notes).
   • If the change affects agent workflows/guardrails, update .cursor/skills/* and/or .cursor/rules/*.

Output format (when you respond as an agent)

• “Files to change” (paths)
• “Key design notes” (real-time constraints + backpressure + fail-safe)
• “Implementation steps”
• “Docs/UX updates” (properties, overlays, Streamlabs guidance)
• “Test plan” (OBS manual recipe + what metrics to watch)
• “Packaging notes” (install paths, signing/notarization if relevant)

Expert OBS knowledge (critical limitations & patterns)

OBS Shader Language (OBS-SL) limitations

OBS uses a custom HLSL-based shader language with strict limitations:

1. No ternary operator (? :) support

   • ❌ float x = condition ? a : b; - NOT SUPPORTED
   • ✅ Use explicit if-else blocks instead
   • This is a common source of "Expected 'name'" compilation errors

2. Limited max()/min() function support

   • ❌ max(float2, float2) - vector overloads NOT SUPPORTED
   • ❌ max(float, float2) - mixed types NOT SUPPORTED
   • ✅ Use component-wise operations: if (a.x > b.x) result.x = a.x; else result.x = b.x;
   • ✅ Scalar max(float, float) and min(float, float) ARE supported

3. No screen-space derivatives

   • ❌ ddx(), ddy() - NOT SUPPORTED (no screen-space derivatives in OBS shader environment)
   • ✅ Use alternative edge detection methods (distance-based, mask gradients)

4. No all() function

   • ❌ if (all(condition)) - NOT SUPPORTED
   • ✅ Use explicit component checks: if (x >= 0.0 && y >= 0.0 && x <= 1.0 && y <= 1.0)

5. Shader file format

   • Must end with proper technique Draw { pass { ... } } block
   • Missing closing braces cause "Unexpected EOF" errors
   • Comments (/* */ and //) are supported

Graphics context requirements

Critical: gs_effect_create_from_file() requires an active graphics context

• ❌ DO NOT call gs_effect_create_from_file() in filter creation (gzfx_create)
  • Graphics context is NOT guaranteed during filter initialization
  • Will fail silently or return NULL with no error details
• ✅ DO use lazy loading in the first render call (gzfx_render)
  • Graphics context is guaranteed active during rendering
  • Use atomic flags to ensure effect loads only once:

    atomic_bool effect_load_attempted;
    if (!filter->effect && !atomic_load_explicit(&filter->effect_load_attempted, ...)) {
        atomic_store_explicit(&filter->effect_load_attempted, true, ...);
        filter->effect = gs_effect_create_from_file(path, &error_string);
        // Get parameters immediately after successful load
    }
    

OBS API patterns & gotchas

1. Recursive API calls

   • ❌ NEVER call obs_source_get_width()/get_height() from within callbacks that OBS uses to compute dimensions
   • This causes infinite recursion and crashes
   • ✅ Use obs_source_get_base_width()/get_base_height() instead for base dimensions

2. Thread safety

   • Render thread: read-only operations only
   • Settings/UI thread: can modify filter state
   • Use atomics for cross-thread communication (not mutexes in hot paths)

3. Effect parameter lookup

   • Must happen after effect is successfully loaded
   • Store gs_eparam_t* pointers in filter struct (they remain valid)
   • Check for NULL parameters before use

4. Error logging

   • gs_effect_create_from_file() can return NULL with or without error details
   • Always check both the return value AND the error string pointer
   • Use blog(LOG_ERROR, ...) for shader compilation errors

Build & packaging specifics

1. CMake generator expressions in scripts

   • ❌ Passing $<TARGET_BUNDLE_DIR:...> directly to shell scripts can cause mangling
   • ✅ Construct paths dynamically in scripts using $(dirname "$0") and relative paths
   • ✅ Or use configure-time variables that are substituted into script files

2. Rpath cleanup

   • Swift linker may add .build directory rpaths that break runtime loading
   • Clean these up in POST_BUILD step using install_name_tool -delete_rpath
   • Script must find plugin binary reliably (use script's location as reference)

3. Plugin bundle structure

   • Binary: PluginName.plugin/Contents/MacOS/PluginName
   • Resources: PluginName.plugin/Contents/Resources/
   • Frameworks: PluginName.plugin/Contents/Frameworks/
   • Use obs_module_file("relative/path") to resolve resource paths

4. Version compatibility

   • OBS plugins are strictly version-locked to libobs version
   • Module compiled with libobs 32.1 will NOT load in OBS 32.0.x
   • Always match OBS version or provide version-specific builds

Performance constraints

1. Render callback timing

   • Must complete in <16ms for 60fps (ideally <8ms)
   • No blocking operations (file I/O, network, heavy CPU work)
   • Prefer GPU operations (Metal shaders) over CPU processing

2. Frame processing

   • Latest-frame-wins: overwrite slots, don't queue
   • Drop frames under load rather than increasing latency
   • Use atomic snapshots for cross-thread data sharing

3. Vision tracking cadence

   • Run at fixed rate (15-30Hz) independent of render FPS
   • Downscale input frames for Vision to reduce processing time
   • Cache results and apply temporal smoothing

Debugging strategies

1. Shader compilation errors

   • Check OBS logs for detailed error messages
   • Error format: (filename, line, column): Error message
   • Common issues: unsupported syntax, missing braces, type mismatches

2. Effect loading failures

   • Check file path (use obs_module_file() for correct resolution)
   • Verify graphics context is active (lazy load in render)
   • Check error string from gs_effect_create_from_file()

3. Runtime issues

   • Use blog(LOG_INFO/LOG_WARNING/LOG_ERROR, ...) for debugging
   • Add counters for FPS, drops, latency, confidence bands
   • Enable debug overlay to visualize ROI/masks in real-time

Additional resources

• More detailed notes: reference.md
• Example response patterns/checklists: examples.md