langbot-testing

# LangBot Testing Use this skill when an agent needs to verify LangBot behavior through the WebUI instead of only reading code. ## Routing - **General WebUI testing**: read `references/web-ui-testing.md`. - **Pipeline Debug Chat**: read `references/pipeline-debug-chat.md`. - **Dify Runner**: read `references/dify-agent-runner.md`. - **Model provider setup or test button**: read `references/model-provider-testing.md`. - **Plugin install/runtime/tool/page smoke**: read `references/plugin-e2e-smoke.md`. - **Local Runner**: read `references/local-agent-runner.md`. - **Local Runner path coverage**: read `references/local-agent-runner-coverage.md`. - **Diff-aware Runner QA after code changes**: read `references/agent-runner-qa-workflow.md`. - **Runner release gate**: read `references/agent-runner-release-gate.md`. - **Sandbox-backed skill authoring**: read `references/sandbox-skill-authoring.md`. - **LangRAG knowledge bases**: read `references/langrag-knowledge-base.md`. - **MCP stdio tool testing**: read `references/mcp-stdio-testing.md`. - **Performance, reliability, or chaos probes**: read `references/performance-reliability-testing.md`. - **Cross-repository workspace and release gates**: read `references/workspace-release-testing.md`. - **Drive a live instance over MCP (not raw HTTP)**: use the `langbot-mcp-ops` skill — the instance exposes an MCP server at `http://<host>:5300/mcp` (reuses API keys). Useful for setting up bots/pipelines/models as test fixtures programmatically. - **Known failures and fixes**: read `references/troubleshooting.md`. - **Reusable test groups**: run `bin/lbs suite list` and `bin/lbs suite plan <suite-id>` before manually assembling a case set. ## Rules - Read `../.env` first and use `LANGBOT_FRONTEND_URL` and `LANGBOT_BACKEND_URL` instead of hardcoded ports. - If a standalone frontend dev server is running, `LANGBOT_FRONTEND_URL` may point to `LANGBOT_DEV_FRONTEND_URL`; otherwise it may point to the backend WebUI. - Confirm the backend and frontend are actually running before testing. - Run `bin/lbs fixture check` before fixture-heavy MCP, RAG, multimodal, or plugin smoke tests. - For runner externalization release checks, run `bin/lbs test run agent-runner-release-preflight` before the full `agent-runner-release-gate` suite so configuration blockers are separated from product failures. - Read `Manual Readiness` in `bin/lbs test plan <case-id>`; `manual_check` means the declared preconditions or setup still need operator confirmation for this run. - Use an authenticated browser profile prepared by `langbot-env-setup`. - Do not expose API keys, OAuth secrets, tokens, or localStorage token values in output. - A WebUI test is not complete until the visible UI result is checked against backend logs or network behavior. - A performance result is not complete without `metrics` evidence and a clear split between LangBot overhead and external provider/tool/network time. - A chaos or reliability result is not complete until the fault scope, cleanup, and recovery checks are recorded. - For a suite, use `bin/lbs suite start <suite-id>` to create the suite evidence root, per-case directories, and `suite-start.json`/`suite-start.md` handoff files; use `bin/lbs test result <case-id>` to write final per-case `result.json`, then run `bin/lbs suite report <suite-id> --evidence-dir <dir>`. - Do not mark a case `pass` until `test result --evidence` covers every value in the case's `evidence_required`. - For runner-specific Debug Chat cases, use the case-specific pipeline env declared by `automation_pipeline_url_env` / `automation_pipeline_name_env`; do not silently reuse a generic `LANGBOT_PIPELINE_URL`.