agentic-harness-evidence-loop

# Agentic Harness Evidence Loop Use this skill when a user, Agent Chat prompt, system event, or CLI request asks why an execution/candidate failed, requests diagnostic evidence, or asks for a repair direction backed by local evidence. ## Phase Boundary Phase 6.8A local collection is always allowed without Control Plane. - Do not call Control Plane write APIs. - Do not read Control Plane tokens. - Do not upload, copy, or migrate raw artifacts. - Do not modify code. - Do not run Git mutation, CI trigger, deploy, rollback, or repair commands. - Do not put raw transcripts, raw logs, raw traces, cookies, kubeconfig, Authorization headers, bearer tokens, host absolute paths, or credential-bearing URLs into summary-safe evidence. Phase 6.8C may perform Control Plane writes only when both conditions are true: - The request runs in a trusted runtime or service-token context. - `agentic-evidence index register` is allowed only when allowed actions explicitly include `index_register`. - `agentic-evidence storage apply` is allowed only when allowed actions explicitly include `storage_apply`. Do not route write requests through agent-runtime. Use the Control Plane runtime API directly from the trusted runtime/service-token context. Do not introduce `CONTROL_PLANE_BASE_URL` or `CONTROL_PLANE_TOKEN` as product-level configuration. Use these inputs according to the spec: - `APAAS_CONTROL_PLANE_RUNTIME_API_BASE_URL` for the Control Plane runtime API base URL. - `APAAS_RUNTIME_CONTEXT_PATH` for the runtime context file when present. - `APAAS_CONTROL_PLANE_TOKEN_PATH` for the service-token file path. - `/run/secrets/control-plane-token/token` as the default service-token file in runtime deployments. - `APAAS_CONTROL_PLANE_TOKEN` only for local/test-only execution, never as product-level configuration. Missing base URL, missing token, forbidden responses, unavailable services, and payload errors must preserve the local bundle. Do not delete, rewrite, or hide local diagnostic artifacts when write-side registration or storage application fails. Never print token values, Authorization headers, cookies, kubeconfig, signed storage URLs, or credentialed URLs. ## Workflow 1. Build or load a `HarnessEvidenceRequest`. 2. Validate the request: ```bash agentic-evidence request validate --request <request.json> ``` 3. Classify and collect local evidence: ```bash agentic-evidence collect --request <request.json> --out <staging-root> ``` 4. Validate the bundle: ```bash agentic-evidence bundle validate --bundle <diagnostic-run-dir> ``` 5. Build or refresh the local repair route: ```bash agentic-evidence repair-route --bundle <diagnostic-run-dir> ``` 6. Explain `harness-feedback.json` and `repair-route.json` to the user using only summary-safe evidence refs. ## Conversation Diagnostic Intake Do not ask the user to write request JSON. When a user or Agent Chat message describes a failure in natural language, build the `HarnessEvidenceRequest` yourself from the available conversation, Builder, execution, diagnostic drawer, and workspace context. Use existing context first: - Prefer explicit workspace_id, task_id, candidate_id, or execution_id from structured request payloads, selected Builder state, current automation evidence, selected diagnostic run, or Agent Chat message actions. - If no subject ref is available, Ask at most one concise follow-up question before running collection. Ask for the smallest identifier that can locate the failed subject, such as the task, candidate, execution, phase, or diagnostic run. - Optional background such as time, page or route, button/action, observed error, recent change, browser/preview URL, and retry count should be summarized into `subject_summary` or `user_message_summary` when available. - Do not block collection waiting for optional background fields when a subject ref is already available. - Do not infer task, candidate, execution, Control Plane URL, token, storage target, or host path from prose when the context does not provide it. For `conversation_implicit`, keep the request local-only unless a trusted structured request explicitly authorizes more: ```json { "trigger_kind": "conversation_implicit", "requested_profiles": [], "allowed_actions": ["collect_summary", "repair_route", "repair_prompt"] } ``` ## Write-Side Workflow Only run each write-side command when its specific Phase 6.8C allowed action is present. Register the diagnostic run index: ```bash agentic-evidence index register --bundle <diagnostic-run-dir> ``` Apply storage decisions: ```bash agentic-evidence storage apply --bundle <diagnostic-run-dir> ``` Explain write results from `manifest.json`, `storage-result.json`, and summary-safe diagnostics only. Do not expose raw payloads, secrets, credential-bearing URLs, or raw artifact contents. ## Trigger Defaults For `conversation_implicit`, default allowed actions are: ```text collect_summary repair_route repair_prompt ``` Only include `collect_debug_only` when the user or trusted system event explicitly allows debug-only local artifacts. `index_register` and `storage_apply` are not active in Phase 6.8A. ## RepairRoute Rules `RepairRoute` is a next-step recommendation only. It can point to `spec_revision_required`, `bugfix_request_required`, `confirmed_implementation_followup`, `agent_chat_inline_candidate`, `human_triage_required`, or `no_action`, but it never authorizes code changes by itself. Repair-route output does not authorize code changes, Git mutation, CI trigger, deploy, rollback, or repair commands.