threadlight-auto

# `threadlight-auto` — Full-auto Threadlight driver ## Presenter-ready completion (explicit opt-in) Set `delivery_profile: presenter-ready` in the existing `specs/manifest.json` only when requested. Follow the single process-owned [presenter-ready contract](../../docs/presenter-ready.md); its absence does not change ordinary experiments. Malformed explicit selection blocks, not off. Run from the complete catalog so the shared evidence consumer is available. The planner adds `presenter_package` before deploy (exact native packaged SDK/framework/adapter checks) and `presenter_ready` at handoff. The coding agent performs only authorized work; zero exit or a successful response cannot replace evidence. The process owner supplies entry -> useful interaction -> persistence where applicable -> independent readback -> reopening, and downloads only if promised. A first-time presenter separately records human acceptance. `presenter_ready` in planner JSON keeps source-ready, deployed, backend-verified, script-verified and human-accepted distinct. It reports supplied, source-bound evidence, not a fresh live attestation. No human acceptance may be auto-generated. Dependency-specific invalidation replaces blanket cascade **only for this profile**: script/sizing changes do not redeploy or replay the backend; runtime, source and new deployment attempts invalidate their actual dependents. Deployable workspace/service source is runtime input, not just presentation. Expired evidence calls for observation refresh, not effect replay. Failed or uncertain backend/deployment evidence stops for reconciliation; expired source blocks new invocation. A failed governed deployment and a missing/failed Safe Check cannot be bypassed by a presenter receipt or a worker's zero exit. Native package proof precedes deployment; selected governance gates retain authority. Handle permission/shared-resource/material-cost/uncertain-effect decisions explicitly, not repeated approvals of routine authorized steps. ## Purpose Replace the manual chain `threadlight-design → threadlight-deploy → threadlight-safe-check → invoke` with a single invocation. Designed for: - **First-timers** who don't yet know which skill fires when - **Demos** where the whole arc has to complete in one Copilot session - **Resumption** when a deploy failed and the operator wants to retry without re-doing earlier stages - **Pilots-from-templates** where the operator just wants to pick `auto-claim-triage` / `credit-memo` / `prior-auth-healthcare` and have the system fill in the boilerplate SEs who already know the per-skill chain should keep invoking those directly — `threadlight-auto` is a wrapper, not a replacement. **Model roles.** Recommend a sufficiently capable authoring model for nontrivial design/build work, respecting the operator's host selection; do not silently change it or global settings. Runtime model/capacity choices remain in Foundation/SPEC, independent of the coding model. At the validated pilot handoff, offer the [optional fixed-artifact runtime comparison](../threadlight-design/references/model-selection.md) when a lower-cost candidate could meet the use case's requirements. This is a manual, explicitly authorized experiment, not a new stage, automatic paid eval, deployment, or hidden downgrade. Keep construction and runtime costs separate. > **Design.** The orchestrator pattern, smart-recovery table, and HARD-STOP > gates are the load-bearing reliability contract of this skill. Threadlight's > stage labels + artifact paths are canonical: the design stage emits > `specs/SPEC.md` + `specs/manifest.json`, and every downstream stage keys off > their hashes. ## Position in the SKILL hierarchy ``` ┌─────────────────────────────────┐ │ threadlight-auto (THIS SKILL) │ ← single entry point │ • parses input prompt │ │ • runs orchestrator.py │ │ • drives sub-skills in │ │ sequence with smart │ │ recovery + resumption │ └────────────┬────────────────────┘ │ via Skill tool ┌─────────────────┬──────┴────────┬──────────────────┐ │ │ │ │ ▼ ▼ ▼ ▼ threadlight- threadlight- threadlight- threadlight- design local-test deploy safe-check (SKILL) (OPTIONAL) (SKILL — runs (gates phase= azd up) post-deploy) │ │ │ │ └────────┬────────┴───────────────┴──────────────────┘ │ each stage benefits from the deploy-time │ failure-mode index F-01..F-22 in │ threadlight-deploy/SKILL.md ▼ ┌──────────────────────────────────────────┐ │ awesome-gbb companion SKILLs │ │ (foundry-hosted-agents, azd-patterns, │ │ foundry-observability, …) │ └──────────────────────────────────────────┘ ``` > **Legs auto does _not_ drive.** Two production-handoff steps > (**`threadlight-cicd`**, **`threadlight-customize`**) and the offline > **`threadlight-router-bench`** *Improve* leg run outside this orchestrator. > `threadlight-auto` is a pilot driver — after a CI run finishes, reach for > `threadlight-router-bench` to harvest a grounded learnings digest (failure > taxonomy + recommendations) and, optionally, a model-router cost/quality > scorecard. It never drives prod-pipeline, customer-onboarding, or offline > self-improvement legs. Selected governance bindings add mandatory gates below. ## Governed actions — mandatory for selected bindings The orchestrator reads the explicit governance contract, without adding bindings, omitting business actions or switching frameworks. Selected bindings require: `design → govern → governed_actions_gate → deploy → governance_probe → safe_check → cost_projection → invoke` | Stage | Required action and exit condition | |---|---| | `govern` | Run `threadlight-govern` with `--emit`; validate the current inventory at `specs/governance-manifest.json`. Offline inventory is not live proof. | | `governed_actions_gate` | Run `threadlight-governed-actions --phase pre-deploy`. Then run `orchestrator.py --workspace <pilot> --complete-stage governed_actions_gate`. This independently validates the complete `governance-ledger/v2` contract and checkpoints exact input hashes; failure blocks deploy. | | `deploy` | Immediately before each actual attempt, run `orchestrator.py --workspace <pilot> --start-stage deploy`. Only after that worker succeeds, run `orchestrator.py --workspace <pilot> --complete-stage deploy`. Failed/interrupted attempts must not be marked complete; retries start a new attempt. | | `governance_probe` | After deploy, use `threadlight-safe-check`'s explicit collector with `.threadlight/governance-probe.json`. It writes `.threadlight/governance-live.json`. Run `orchestrator.py --workspace <pilot> --complete-stage governance_probe` to validate and publish its unchanged nested manifest to `specs/governance-manifest.json`; shared validation must pass before any subsequent stage. | Run the planner before **each** stage and execute only the first runnable stage. Never treat pending stages as permission to continue past a failed gate. The programmatic `execute(workspace, worker)` driver enforces ordering and revalidates artifacts after worker completion. Nonzero exit, missing tooling, malformed evidence, legacy v2, or a local assessment labeled green cannot unlock the live gate. On resume, changed contract/manifest/image/policy/configuration or expired proof invalidates the corresponding checkpoint and downstream stages. Every actual deployment invalidates earlier proof, even for an unchanged image or version. `.threadlight/governance-execution-state.json` records the attempt ID, execution times, input fingerprint and accepted collection digest. Collection must start after that exact successful attempt; file modification times are not proof. The programmatic driver records these transitions automatically. Do not edit collector timestamps or cloud history to satisfy a checkpoint. The pre-deploy fingerprint covers normalized selection, policy/configuration, scope, source files and actual host wiring. Parent runtime FQDN, image digest and agent-version outputs do not invalidate that authorization; completed attempts separately bind those outputs for subsequent Task11 verification. Source, tools, policy, scopes and non-output configuration changes still require a new gate. Interrupted/failed attempts force an actual deploy retry even if an old FQDN exists. A successfully completed unchanged attempt may resume its proof without redeploying. With no governance selection, the legacy `govern` worker remains advisory and cannot assert live enforcement; malformed selections remain blocked. Contract JSON, parent manifest declarations and SPEC YAML use the shared contract validator. Malformed declarations, unsupported mode aliases, duplicate keys or conflicting mirrors produce `invalid-governance-configuration` and block all workers; they never select the legacy/off path. A human-readable selection without a complete supported contract is unresolved, not permission to deploy. Only the **explicitly reserved staging noop**, at its exact binding and `pre_tool_call` path, is safe to probe. Never probe business actions or production canaries to clear a gate. Its receipts cannot certify another tool, lifecycle point, agent or deployment. Unproven business bindings remain blocked. For an explicitly off or unbound-only valid contract no governance stages are scheduled. Without selected bindings, the existing consequential-action advisory handoff remains available with the following wording: | Key | Recommendation | |---|---| | `execution` | `manual-explicit` | | `design` | `run threadlight-governed-actions --phase design after threadlight-design` | | `pre_deploy` | `run threadlight-governed-actions --phase pre-deploy before deploy` | | `post_deploy` | `run threadlight-governed-actions --phase post-deploy against staging only` | | `manifest` | `tests/governed-actions-manifest.json` | The handoff appears only when the workspace declares a consequential action (`specs/manifest.json` `consequential_actions: true`, or a root action registry declaring a non-read `consequence`) or already carries a manifest — an unrelated pilot is never nagged. **Manifest summary (`governed_actions_manifest`).** When `tests/governed-actions-manifest.json` exists, `summarize_governed_actions_manifest()` reads it as untrusted JSON and checks only what auto can check for itself: schema `threadlight-governed-actions-manifest/v1`, a known lifecycle phase, a clean source bound to the checked-out commit, freshness, and summary counts that agree with the findings they claim to summarise. A schema-valid, fresh, commit-bound manifest is reported as `summarized` with its verdict and per-status counts. Anything else — stale, expired, dirty, mis-bound, miscounted, malformed — is reported as `rerun-recommended` with zeroed counts and no verdict, and the recommendation is the same approved lifecycle wording. **Trust limitations.** The advisory summary is a *read*, not a verification. The mandatory gate separately reuses `threadlight-production-ready`'s strict validator, which re-derives policy hashes, resolves evidence and validates mediation paths when it folds the manifest into `AGT-007` / `HITL-008` / `SUP-014`. Auto's `summarized` status therefore means "this artefact is well-formed, current, and bound to this commit", never "this pilot is governed". Production rollout remains human-owned. ## Optional AgentOps stage After invocation and before eval/red-team consumers, `agentops` invokes `threadlight-agentops` only for agent roots containing `agentops.yaml`. The stage is read-only: no native eval or Doctor is started by Auto, and no AgentOps installation/configuration is performed. It emits `specs/agentops-manifest.json` from existing artifacts. Missing opt-in always skips, including after an upstream cascade. Missing, malformed, expired or changed evidence requires normalization; a current valid `partial` or `blocked` manifest is reused without rerunning native operations to seek green. Selected governance bindings still require `governed_actions_gate` before deploy and current `governance_probe` before subsequent stages, including AgentOps. AgentOps evidence cannot satisfy either gate. New eval/Doctor execution is a separate explicit, scoped `foundry-agentops`/`threadlight-cicd` handoff, not automatic recovery. Production-ready remains the final advisory owner. ## Input parsing `threadlight-auto` accepts two input shapes; freeform is the default. ### Freeform (default) A single natural-language prompt. Examples: - `"Build me an auto-claim triage agent for Contoso Mutual in acme"` - `"Run threadlight-auto with the credit-memo scenario, customer=Contoso Financial, env=dev"` - `"Scaffold a prior-auth pilot for Northwind Health, tenant=acme, region=westus3"` Parsing rules: - **Scenario template** — look for `with the <name> scenario` or `scenario=<name>` (default: `auto-claim-triage`) - **Customer name** — look for `for <Name>` or `customer=<Name>` (default: derived from scenario) - **Tenant alias** — look for `tenant=<alias>` or `in <alias>`, default to `~/.azure-tenants/index.json` `default_alias` - **AZD env** — look for `env=<name>`, default `dev` - **Region** — look for `region=<name>` or `in <region>`, default `westus3` (auto-fallback to `eastus2` → `northcentralus` on quota fail) - **Workspace dir** — derived from `<customer-slug>-<scenario>`, written to `~/Repos/<slug>/` if not specified ### Structured (power-user override) ``` Use threadlight-auto with: scenario: auto-claim-triage customer: Contoso Mutual tenant: acme env: dev region: westus3 workspace: ~/Repos/contoso-claim-triage ``` ### Kratos-export entry path (start from an exported bundle) `threadlight-auto` has a **second entry path** alongside the freeform/structured "from-scratch" flow above: starting from a **Kratos-exported project**. It is selected automatically when the workspace already contains a Kratos export (`src/hosted-agent/` **and** `use-cases/<x>/` — see [`docs/KRATOS-BRIDGE.md`](../../docs/KRATOS-BRIDGE.md)), or explicitly: ``` Use threadlight-auto with: mode: kratos-export workspace: ~/Repos/wealth-management-agent # unzipped <use-case>-foundry-agent.zip tenant: acme env: wealth-management-prod ``` In this mode the orchestrator **does not run Design (stage 1)** and **does not regenerate runtime files in Deploy (stage 3)** — Kratos already shipped a deployable project. The chain becomes: ``` Stage 0 Preflight → (deploy: enrich/validate only + backfill evals/ — threadlight-deploy Kratos-export mode) → azd up (if not already deployed) → Safe-check (post-deploy, accepts trimmed infra) → Cost-projection (discover from export infra/) → Invoke → Production-ready (advisory; trimmed infra = informational) ``` The skills root resolves to `use-cases/<x>/skills/` for every stage. Optional extension skills (`threadlight-hitl-patterns`, `threadlight-event-triggers`, `threadlight-workspace-ui`, `citadel-spoke-onboarding`) run on demand, writing next to the existing use-case skills. ## Stage 0 — Preflight **Always runs** as the bootstrap preflight. Checks: 1. Tenant + subscription match `~/.azure-tenants/index.json` for the alias (azure-tenant-isolation rule 4a) 2. Tool versions: `az ≥ 2.86`, `azd ≥ 1.25.4`, `bicep ≥ 0.43`, `uv ≥ 0.7`, `node ≥ 22`, `python ≥ 3.12` 3. `azd ai agent` extension installed in the alias's `AZD_CONFIG_DIR` 4. The `../threadlight-design/references/runtime-policy.json` dependency must **always be readable**. If `threadlight-design` is not installed/enabled, **HARD STOP** and tell the operator to install or enable it — never fall back to a remembered default. Selector validation then depends on the input path: - **Resume / hand-crafted path (complete foundation):** when `specs/foundation.md` exists, validate the record using the **same complete-foundation rules as Deploy Runtime-policy pre-flight step 5**. The tuple must be compatible; a concrete `policy_route` must exactly match its declared selectors and be the first matching route for the current signals; and `explicit-supported-choice` requires operator provenance `source: provided`, no active `blocked_when` signals, and an empty `unresolved_signals` list (`requires_resolved_signals`). If Foundation exists but SPEC § 11e is **not yet present** because Design stopped between those writes, **do not hard-stop or reject** the resume: validate the Foundation internally using `runtime_shape` plus its capability signals, and **defer only the mirror cross-check** until Design resumes or Deploy runs after SPEC exists. A **legacy foundation.md** that is missing `protocol`, `policy_route`, or `capability_signals` is not complete: do not reject or default it in Stage 0; **defer migration and final validation to Stage 3 Deploy** (Runtime-policy pre-flight steps 2–3). - **Greenfield path:** `specs/foundation.md` does not exist yet; Stage 1 Design creates it. Do not hard-stop in Stage 0. **Defer** selector resolution to Design and enforce validation again at Stage 3 Deploy. - **Kratos-export mode:** **skip foundation/selector validation** because the exported runtime is preserved verbatim and deploy Phase 2 is skipped; only the unconditional policy-dependency readability check above applies. `threadlight-auto` owns **no separate framework or protocol default**. 5. Writes `.threadlight/preflight-passed.json` with `foundation_sha256: <sha256>` when Foundation exists, or `foundation_sha256: null` before it exists. The marker has 24h maximum validity, but Foundation creation, editing, or removal invalidates it immediately. > **🛑 HARD STOP #1 — Tenant assertion failure.** If tenant verification fails (wrong > tenant or wrong subscription active), `threadlight-auto` STOPS IMMEDIATELY. No > auto-recovery. Money is about to be spent in the wrong place — operator must > fix isolation before retrying. > **Runtime-policy contract.** `threadlight-design` and `threadlight-deploy` > both inherit `../threadlight-design/references/runtime-policy.json`. > `threadlight-auto` never invents a competing default. A missing/unavailable > `threadlight-design` dependency is a Stage-0 hard stop; greenfield selector > validation begins after Design creates the foundation, Kratos-export mode > preserves its supplied runtime, and Deploy remains the final policy gate. > Canonical default tuple: `github-copilot-sdk` + `agent` + `invocations` (`policy_route: default-agent`). ## Resumption — read `.threadlight/auto-state.json` first `.threadlight/auto-state.json` is owned by the `threadlight-auto` guidance contract. The Python planner (`references/orchestrator.py`) reads it (if present) to compute which stages are already done; it does **not** write or migrate that file. With `--commit`, the planner writes `.threadlight/auto-next.json` for the coding agent instead. Stages are skipped when ALL conditions hold: | Stage | Skip when | |---|---| | Preflight | `.threadlight/preflight-passed.json` exists AND is `< 24 h` old AND its `foundation_sha256` matches the current Foundation (including `null` while absent) | | Design | `specs/SPEC.md` exists AND `sha256(SPEC.md) == auto-state.json[design].artifact_hash` AND no `[NEEDS CLARIFICATION:` markers | | Local-test | `specs/SPEC.md` exists AND `src/agent/main.py` runs locally (optional stage; skipped on freshness if SPEC unchanged) |