| name | zworkforce-zarvis-voice-ui |
| description | Implement and review zWorkforce Z.A.R.V.I.S. browser voice UI, push-to-talk, realtime audio, animated orb state, and voice BFF changes without weakening auth, secret, approval, or accessibility boundaries. |
zWorkforce Z.A.R.V.I.S. voice UI
Use this skill when changing the root zWorkforce dashboard voice card, ZVoice, browser audio capture/playback, voice session bootstrap, realtime event handling, or voice-related static assets.
Required reading
Read the narrow relevant files before editing:
AGENTS.mdpackages/zarvis/AGENTS.mdROADMAPS.mdexec-planning-zarvis.mdpackages/zarvis/docs/architecture/openjarvis-upgrade-map.mdpackages/zarvis/docs/architecture/voice-agent.mdpackages/zarvis/apps/zvoice/README.mdpackages/zarvis/apps/zvoice/public/app.jspackages/zarvis/apps/zvoice/server.mjszworkforce/static/index.htmlzworkforce/static/app.jszworkforce/static/styles.css
Architecture rules
- Reuse the existing ZVoice realtime protocol and voice services. Do not create a parallel record/upload assistant stack unless a documented fallback explicitly requires it.
- Do not iframe ZVoice by weakening its defensive frame policy. Build a dashboard-native presentation using shared browser-safe client logic.
- Browser code MUST NOT receive provider keys, service tokens, edge secrets, database credentials, GitHub tokens, or internal action credentials.
- Tenant and subject authority comes from authenticated server-side context. Never trust arbitrary client-supplied identity as authorization.
- Spoken text is input, not mutation approval. Approval-required actions remain behind the existing task/action approval contracts.
- Keep microphone capture visibly user-controlled. Stop tracks, AudioNodes, sockets, timers and queued playback on cancellation, navigation, disconnect and error paths.
- Preserve interruption/barge-in behavior.
- Static assets must be compatible with the repository CSP and must not require unsafe inline script/eval.
Canonical voice state
Use one presentation-independent state model:
idle
arming
listening
transcribing
thinking
approval_required
speaking
interrupted
muted
error
Do not infer security state from animation state. Authorization/approval comes from server responses/contracts.
Push-to-talk requirements
- Pointer/touch press starts PTT after microphone/session readiness checks.
- Pointer release/cancel ends the turn.
Space may act as hold-to-talk only when focus is outside editable/form controls.- Ignore keyboard auto-repeat.
- Provide a click/toggle fallback for accessibility.
Escape cancels assistant playback/current response, not an approval record.- Prevent multiple simultaneous capture sessions.
- Show permission denied, backend unavailable, disconnected and retry states explicitly.
Animated orb requirements
The orb is a state visualization, not decoration-only UI.
idle: low-energy ambient motion.arming: bounded pulse/spinner.listening: microphone-level/reactive expansion where available.transcribing: directional scan or restrained activity.thinking: orbital/processing motion.speaking: response-reactive waveform/glow where available.approval_required: stable warning state; do not mimic success.error: explicit text/icon plus visual state.
Respect prefers-reduced-motion; all critical information must also be available as text/live-region state.
Implementation workflow
- Identify whether logic belongs to presentation, shared browser voice client, zWorkforce BFF, zvoice server, voice gateway, or voice agent.
- Prefer extracting reusable protocol/audio logic from ZVoice over copying it into the dashboard.
- Add the narrow test before or with behavior changes.
- Test cleanup and failure paths, not only successful speech.
- Inspect generated/static output for credential names, tokens or internal URLs that should not be exposed.
- Update voice architecture/operations docs for contract/config changes.
Required test cases
Cover as applicable:
- microphone unsupported/denied;
- PTT press, release, pointer cancel and keyboard lifecycle;
- session/ticket rejection and expiry;
- WebSocket connect/error/close;
- speech started/stopped;
- partial/final transcript handling;
- assistant audio/text response;
- barge-in/cancel;
- reconnect/cleanup;
- approval-required state;
- tenant/auth denial;
- upstream timeout/failure;
- reduced-motion/accessibility hooks;
- no server secret in browser payload/static assets.
Validation
Run the smallest relevant tests, then:
python -m compileall -q zworkforce tests
PYTHONPATH=. python -m unittest discover -s tests -v
pnpm --dir packages/zarvis test
If voice runtime dependencies or containers change, also run the documented voice service health/smoke checks. GitHub Actions remains the final release gate.
