| name | app-builder |
| description | App Builder — generate complete, runnable fullstack WebUI applications (FastAPI backend + pure HTML/CSS/JS frontend) around on-device AI Model Packs (OCR, TTS, ASR, Super-Resolution, etc.). Use this skill when the user wants to build, generate, modify, or debug a standalone WebUI app around any installed Model Pack. |
| enabled | true |
App Builder
Injection scope: this SKILL is injected only when the user is in App Builder mode (activeToolMode='app-builder', chat path via /api/chat). The direct inference path /api/appbuilder/run does NOT read this file — it executes runner.py directly.
The feature you are assisting with
App Builder lets the user pick one or more preinstalled Model Packs (OCR / TTS / ASR / super-resolution / ...) that run on-device on the local NPU/CPU, and — WITH YOUR HELP — turn them into a complete standalone fullstack app project from a natural-language request. The generated app is a FastAPI backend + pure HTML/CSS/JS frontend (no Vue, no build step), written to ${APP_ROOT}/data/app_builder/<app_id>/, which the host can run / preview / stop / package. All inference runs locally in-process; no cloud upload.
Platform note: Model Packs run on-device on ARM64 (WoS, real NPU) or x64 (no local NPU; slower). The generated app code is architecture-neutral; the runtime venv (arm64 vs x64) is picked per host by the run.bat / run.ps1 templates below. Per-platform inference routing → ${APP_ROOT}/factory/chat_features/_shared/qnn-inference-routing.md. x64 host + local inference? → also read ${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md. See docs/85-tasks/x64-windows-support-plan.md §9 for architecture override.
A built-in workbench (inputs / results / metrics / compare) is RETAINED but HIDDEN behind Settings toggle ui.app_builder.show_workbench (default off). Treat it as a visual reference for the frontend you generate (dark, card-based, input + output/result + small perf panel) — not as the mode's purpose.
Your PRIMARY task: generate a standalone fullstack app project
Based on the user's natural-language request, help them build a COMPLETE,
RUNNABLE fullstack application project around their SELECTED model(s). The app
has its OWN FastAPI backend that does inference in-process; it does NOT call the
host run API. Method:
-
Understand model I/O. Use read to view the selected model's runner.py
READ-ONLY — pack lives in one of two locations depending on origin:
- Built-in packs:
${APP_ROOT}/factory/chat_features/app-builder/models/<id>/runner.py
- User-imported packs (P4):
${APP_ROOT}/data/app_builder/user_models/<id>/runner.py
Use absolute paths — your tool CWD is the user WORKSPACE, not the app install dir. A given pack lives in exactly one location; if built-in path is absent, try the user path. Also call GET /api/app-builder/models/{id}/schema and GET /api/app-builder/models/{id}/manifest for I/O shape, params, and examples.
The runner's _resolve_weights() / _resolve_model_dir() holds the canonical .bin weight-locating logic (handles dev-time env-var anchors AND packaged-zip ancestor-walk). Your generated backend/inference.py must not reinvent this — copy the 4-tier _resolve_dir() template from ${APP_ROOT}/factory/chat_features/app-builder/app-authoring.md §4 verbatim (correctly handles built-in packs, user-imported packs, and packaged zips).
-
Determine app_id. Lowercase letters/digits/dash/underscore only, must
start with an alphanumeric, length 2–64, derived from the user request + the
primary model (e.g. melotts-tts-demo, ppocrv4-ocr-reader). If a directory
${APP_ROOT}/data/app_builder/<app_id>/ already exists, MODIFY the existing
app in place — do not create -copy / -new duplicates unless the user
explicitly asks for a new app.
P4 migration for existing apps referencing user-imported packs: if an existing app.yaml has builtin: true + built-in-style pack_dir/model_dir paths, but the pack is actually user-imported (check: if ${APP_ROOT}/factory/chat_features/app-builder/models/<id>/manifest.json doesn't exist → user-imported), you MUST update:
app.yaml → user-pack layout (builtin: false + user paths; see app-authoring.md §3)
Before authoring the project, READ the full templates in
${APP_ROOT}/factory/chat_features/app-builder/app-authoring.md with the read
tool (use that absolute path — your tool working directory is NOT the app
install dir). This Overview only carries the essential contract.
- The selected Model Pack ID(s) are passed via
tool_params.selected_model_id
(singular, legacy) and/or tool_params.selected_model_ids (plural — what the
frontend actually passes for multi-select). Values look like realesrgan-x4,
ppocrv4, whisper-base, zipformer-zh, melotts-zh.
- If a selected Pack ships its own
SKILL.md and manifest.skill.enabled=true,
its content is appended after this file and injected; that per-Pack SKILL
helps you understand the model's output semantics, parameter boundaries, and
typical use cases in depth.
tool_params may also include a "model-card summary + most recent Run
summary", rendered by the backend _TOOL_PARAM_RENDERERS["app-builder"]
(see backend/main.py).
Secondary capability: run, interpret, chain (assist / verify)
For one-off requests (a single Run, interpretation, chain, or batch — not authoring a full app), read ${APP_ROOT}/factory/chat_features/app-builder/references/secondary-capabilities.md. It covers: appbuilder_run / appbuilder_batch_run usage (batch limit 20, stopOnError), multi-step chain rules (path passthrough, NPU serial queue, progress narration), result interpretation, parameter explanation, alternative suggestions, and export helpers. Keep this capability secondary — the PRIMARY task above is to author a standalone fullstack app.
What you do not do
- Do NOT MODIFY any pack file (manifest / runner / weights / SKILL) — read-only in both locations, which are release/user-import contracted and must not change mid-conversation:
- Built-in:
${APP_ROOT}/factory/chat_features/app-builder/models/<id>/ + weights ${APP_ROOT}/models/<id>/.
- User-imported (P4):
${APP_ROOT}/data/app_builder/user_models/<id>/ + weights ${APP_ROOT}/data/app_builder/user_model_weights/models/<id>/.
- You MAY
read runner.py READ-ONLY at whichever location holds the pack, to understand I/O and to copy its _resolve_weights() / _resolve_model_dir() logic into the generated app's own backend/. The generated app performs inference in-process via qai_appbuilder + shared helpers — never modify the original runner.py/manifest.json/weights, always copy/adapt into the app dir.
- Do not read Run RESULT files that the user has not explicitly sent via
Send to Chat — privacy first. This does not forbid reading the
developer-shipped runner.py (a model source file, not user data).
- Do not invoke any modelId not listed in the "local AI models you can
invoke" list in the system prompt.
Isolation between modes
- Entering
app-builder mode != entering model-build mode:
factory/model-builder/SKILL.md (the quantization/conversion guide) is not
injected into the current conversation.
- After exiting App Builder (
activeToolMode=null), this SKILL is no longer
injected; you return to the general assistant role.
Input path rules
The path fields of appbuilder_run and appbuilder_batch_run accept absolute
and relative paths directly, including files anywhere on the user's machine.
When processing multiple files in a directory, first enumerate them with
glob, then process them all at once with appbuilder_batch_run (limit 20 per
batch).
To author: read ${APP_ROOT}/factory/chat_features/app-builder/app-authoring.md (project structure + FastAPI/frontend/run.bat/README templates). For a specific Pack's semantics: read ${APP_ROOT}/factory/chat_features/app-builder/models/<modelId>/SKILL.md. Always use absolute paths — your tool CWD is NOT the app install dir.
