test-voice-layer-app

name	test-voice-layer-app
description	Build and exercise the packaged Voice Layer macOS application through Codex Computer Use. Use for native UI acceptance, local Rig or llama-server chat checks, accessibility and keyboard verification, 1200x900 visual inspection, session persistence checks, and real voice or TTS acceptance where device testing is explicitly required.

Test Voice Layer App

Validate the real packaged Tauri application, not only its web frontend or build output. Keep shell, native UI, model-server, microphone, and playback evidence separate.

Boundaries

• Read the repository AGENTS.md and record git status --short --branch before testing.
• Preserve existing sessions and source changes. Create a fresh test session rather than altering an older conversation.
• Treat llama-server as user-managed. Check it, but do not start, stop, replace, or reconfigure it unless the user explicitly asks.
• Do not request microphone permission during startup. Exercise the microphone only when real voice acceptance is in scope.
• Do not claim microphone transcription or audible TTS from builds, DOM state, or screenshots. Those require real device evidence.
• Expect packaged UI testing to add a local chat session in the app's WebView storage. Disclose this retained test data; never delete existing sessions.
• Never use Computer Use to control Codex itself. It intentionally refuses com.openai.codex; leave task approvals to the user.

1. Establish the baseline

Run the supported checks from the repository root:

git status --short --branch
just check
just bundle

When local LLM chat is in scope, verify the existing server before opening the app:

curl --fail-with-body --silent --show-error http://127.0.0.1:8080/health
curl --fail-with-body --silent --show-error http://127.0.0.1:8080/v1/models

Confirm that /v1/models advertises the alias expected by the current Rust integration. Inspect the source rather than assuming that alias remains glm47-flash.

The packaged app should be at:

<repo>/src-tauri/target/release/bundle/macos/Voice Layer.app

Build again whenever source or bundle inputs change. A previously running process is not evidence for the newly built bundle.

2. Load Computer Use

Read and follow the available computer-use:computer-use skill before operating the app. Use node_repl for every UI action.

Derive <computer-use-plugin-root> from that skill's current path by removing /skills/computer-use/SKILL.md; do not pin a cached plugin version in this project skill.

Initialize the persistent Node session once:

if (!globalThis.sky) {
  var { setupComputerUseRuntime } = await import(
    "<computer-use-plugin-root>/scripts/computer-use-client.mjs"
  );
  await setupComputerUseRuntime({ globals: globalThis });
}

Use var for reusable top-level bindings because the Node session persists across calls.

3. Open and inspect the packaged app

Use an absolute bundle path:

var voiceLayerBundle =
  "<repo>/src-tauri/target/release/bundle/macos/Voice Layer.app";
var voiceLayerState = await sky.get_app_state({
  app: voiceLayerBundle,
  disableDiff: true,
});
nodeRepl.write(voiceLayerState.text);

From the accessibility tree, verify:

• the standard Voice Layer window;
• Chat navigation and New task;
• an editable Ask for what you want text field;
• the microphone control's current accessible label;
• the conversation and live-status surfaces;
• a default 1200x900 window when visual acceptance requires that viewport.

After every UI action, call get_app_state again and derive fresh element indexes. Never reuse an index from stale accessibility output.

4. Exercise a deterministic local chat

Click New task using its current accessibility index. The app may reuse an existing empty session, which is expected.

Re-read the full state, locate the composer, and submit:

Reply with exactly RIG_STREAM_OK and nothing else.

Use accessibility actions rather than coordinates:

await sky.set_value({
  app: voiceLayerBundle,
  element_index: COMPOSER_INDEX,
  value: "Reply with exactly RIG_STREAM_OK and nothing else.",
});
await sky.press_key({ app: voiceLayerBundle, key: "Return" });
voiceLayerState = await sky.get_app_state({
  app: voiceLayerBundle,
  disableDiff: true,
});
nodeRepl.write(voiceLayerState.text);

Accept either a busy state or a completed state on the first refresh because fast local models can finish before capture. Continue refreshing only while the UI shows meaningful progress.

For a completed turn, require:

• the user message is present;
• one visible assistant answer is present;
• the visible answer is RIG_STREAM_OK;
• no reasoning_content or hidden reasoning appears;
• the local completion status is announced;
• submitting the user message does not start TTS;
• only the completed assistant answer starts TTS;
• focus returns to the composer;
• the microphone remains the only other composer control.

Submit a second turn:

What exact token did you just reply with? Reply with only that token.

Require RIG_STREAM_OK again. This is behavioral evidence that prior user and assistant turns crossed the TypeScript-to-Rust boundary.

5. Inspect the rendered result

Emit the latest screenshot:

var voiceLayerFs = await import("node:fs/promises");
var voiceLayerUrl = await import("node:url");

if (voiceLayerState.screenshot) {
  await nodeRepl.emitImage({
    bytes: await voiceLayerFs.readFile(
      voiceLayerUrl.fileURLToPath(voiceLayerState.screenshot.url)
    ),
    mimeType: "image/png",
  });
}

Inspect the actual pixels for:

• distinct user and assistant alignment;
• readable message wrapping and contrast;
• no clipped chat panel, composer, or sidebar;
• the composer and microphone fitting the 1200x900 window;
• no visible hidden-reasoning text;
• sensible scroll position after the streamed answer.

Accessibility output and screenshots prove different things; retain both.

6. Verify relaunch persistence

Quit the packaged app and reopen the exact bundle:

await sky.press_key({ app: "com.wcygan.voice-layer", key: "super+q" });
voiceLayerState = await sky.get_app_state({
  app: voiceLayerBundle,
  disableDiff: true,
});
nodeRepl.write(voiceLayerState.text);

Verify the test session title and both user/assistant turns remain after relaunch. Emit another screenshot when accessibility output collapses the conversation's descendants; absence from a partial tree is not proof that messages disappeared.

7. Handle Computer Use recovery

• If Computer Use says the bundle changed after just bundle, re-query the latest state before another action.
• If the bundle path stops resolving, retry with bundle identifier com.wcygan.voice-layer.
• Call list_apps() only after name/path lookup fails, then use the returned bundle identifier.
• For a transient ScreenCaptureKit -3811 capture failure, fetch a fresh state once using the bundle identifier. Do not enter an unbounded retry loop.
• If UI action and source state no longer match because another writer changed the checkout, stop and re-establish ownership and the bundle baseline.

Do not manufacture failure recovery by stopping the user's server. If the server is already unavailable, verify the app shows an actionable error and returns focus to an editable composer. Otherwise, report that the unavailable-server UI was not exercised and rely only on focused offline tests that actually cover that path.

8. Report evidence

Report these acceptance dimensions independently:

• baseline and final git status;
• just check and just bundle exit results;
• server health and advertised model alias;
• accessibility evidence for entry, completion, focus, and control labels;
• screenshot evidence at 1200x900;
• deterministic first-turn and history-dependent second-turn results;
• evidence that submitted prompts remain silent and completed replies trigger TTS;
• relaunch persistence;
• microphone-to-transcript evidence, if actually exercised;
• audible playback evidence, if actually exercised;
• untested failure states and retained local test-session data.

Never turn a build, unit test, screenshot, or status string into a broader claim than it proves.