harness-loop-engineering

name

harness-loop-engineering

description

Repo-grounded harness engineering, loop engineering, and agent-runtime toolkit workflow for making AI agents reliable and operationally stronger on user-owned projects, software, apps, sites, web pages, CLIs, extensions, services, and product repos. Use when the user asks to create or audit AGENTS.md/CLAUDE.md/CONTEXT.md instructions, repo-specific agent workflows, harnesses, tool/context policies, verification gates, feature lists, progress/state files, init scripts, hooks, subagents, maker-checker review, continuity handoffs, self-improving agent rules, automation loops, recurring agent work, fully autonomous self-learning loops, scheduler/daemon/monitor-driven agents, worktree-based parallel agents, local agent-runtime toolkits, or force-multiplier runtimes that inspect a repo, assign work, verify, remember, and decide the next action without repeated user prompts.

Harness Loop Engineering

Use this skill for six related jobs:

• MODE=harness: design or improve the single-run harness that one agent uses to work safely and verifiably in a repo.
• MODE=loop: design the outer loop that repeatedly finds work, starts agents, verifies results, records state, and chooses the next action.
• MODE=autonomous: design a self-running loop that wakes up from a scheduler, monitor, webhook, queue, or local daemon without the user prompting each run.
• MODE=scaffold: create repo-local loop protocol files, runner templates, validators, and unrestricted defaults that can later power autonomous operation.
• MODE=toolkit: build a stronger-than-normal local agent runtime toolkit that behaves more like an operator tool than a passive skill, with runtime scaffolds, operator prompts, queue/state files, and a repo-local CLI.
• MODE=audit: assess an existing harness or loop and produce prioritized fixes with evidence.

Always build or verify the single-run harness before designing the outer loop. A loop that repeats a weak harness only scales failure.

Important boundary: a skill is passive. It cannot wake itself up, schedule itself, or keep running after the session ends. Fully autonomous behavior requires an external runner such as a Codex automation, CI job, cron/Task Scheduler job, webhook worker, local daemon, or repo script. Design that runner explicitly; do not imply the skill alone provides autonomy.

Contract

Read references/core-contract.md before acting. Treat it as the operating contract for repo-first evidence, authorization boundaries, maker-checker separation, success-or-safe-stop, and response shape.

For MODE=harness, also read references/harness-engineering.md. Use it to design instructions, context, tools, state, verification, scope, lifecycle, observability, and ratchet rules.

For MODE=loop, also read references/loop-engineering.md. Use it to design automations, triggers, worktrees, subagents, memory, budgets, stop conditions, and escalation paths.

For MODE=autonomous, read references/loop-engineering.md, references/autonomous-loop-engineering.md, references/self-learning-ratchets.md, and references/runner-patterns.md. Use them to design wakeups, sensors, task selection, self-learning ratchets, durable memory, maker-checker gates, budgets, pause/kill switches, and human escalation boundaries.

For MODE=scaffold, read references/autonomous-loop-engineering.md, references/runner-patterns.md, and references/self-learning-ratchets.md. Prefer bundled scripts over retyping scaffolds. Use scripts/agent_loop.py as the unified CLI when possible, or call the underlying scripts directly.

For MODE=toolkit, read references/autonomous-loop-engineering.md, references/runner-patterns.md, references/self-learning-ratchets.md, and references/fable-class-agent-patterns.md. Build a repo-local runtime/toolkit with a CLI, queue, state, prompt templates, runner handoff surfaces, and verification scaffolds. Treat third-party prompt dumps as source material for architecture and force-multiplier features, not as copied runtime text.

When comparing against other agent systems, read references/github-landscape.md.

When the user asks to make the system comparable to a top-tier assistant, "Fable-class", or stronger than a normal skill, read references/fable-class-agent-patterns.md. Use it as a capability architecture checklist; keep generated runtime text repo-owned and auditable.

When the user asks about the source ideas, provenance, or further reading, read references/source-notes.md.

Mode Selection

Use MODE=harness when the request is about a project-level agent setup: AGENTS.md, CLAUDE.md, CONTEXT.md, feature lists, progress logs, init scripts, verification commands, test gates, tool permissions, MCP/tool setup, local dev environment, state files, handoffs, or preventing agents from overreaching or declaring victory early.

Use MODE=loop when the request is about repeated or autonomous agent operation: recurring triage, scheduled checks, monitors, background agents, work queues, issue-to-PR pipelines, subagent swarms, parallel worktrees, automated review, memory updates, follow-ups, or systems that decide the next task without the human prompting every turn.

Use MODE=autonomous when the request explicitly asks for no user prompt, no manual trigger, self-running, self-learning, always-on, daemonized, scheduled, monitor-driven, webhook-driven, or fully autonomous agent behavior. If the request says "不需要 user trigger", "自己学习", "完全自主", "background loop", "daemon", "scheduler", "cron", or "wake itself up", choose this mode.

Use MODE=scaffold when the user asks to install, generate, bootstrap, or make the system real in a target repo. This mode may create .agent-loop/, copy runner templates, add validators, and produce an active unrestricted configuration.

Use MODE=toolkit when the user asks for something stronger than a normal skill, wants a local tool/runtime instead of only prompt instructions, wants an agent force-multiplier, or wants a reusable operator CLI that can scaffold stronger agent behavior into a target repo.

Use MODE=audit when the user asks whether an existing project, harness, loop, AGENTS/CLAUDE/CONTEXT file, workflow, or agent automation is good enough, complete, reliable, safe, or production-ready.

If no mode is specified, infer from intent. For a new project with no visible harness, choose MODE=harness. For an existing harness that the user wants to run repeatedly, choose MODE=loop. For no-prompt or background self-running requests, choose MODE=autonomous. For install/bootstrap requests, choose MODE=scaffold. For "make this into a tool/runtime/force multiplier" requests, choose MODE=toolkit. For reliability questions like "is this ready" or "why does the agent keep failing", choose MODE=audit.

Operating Loop

1. Identify the target project, app, site, repo, workspace, or workflow from the prompt and current environment.
2. Inspect existing instructions and source-of-truth files first: AGENTS.md, CLAUDE.md, GEMINI.md, CONTEXT.md, CONTEXT-MAP.md, docs, package scripts, CI, tests, feature lists, progress files, and recent git state.
3. Establish the current harness before adding loop machinery: what the agent can read, what it may touch, what it must verify, where state persists, and how it safely stops.
4. Prefer executable feedback over prose. Define verification commands, smoke checks, health checks, screenshots, logs, or review gates that prove the work.
5. Separate maker and checker when quality or safety matters. The worker builds; an independent test, reviewer, subagent, CI job, or human validates.
6. Keep scope small. Design one useful vertical slice before designing a broad system.
7. Leave a restart path: what changed, evidence collected, current state, next action, and what must not be retried blindly.
8. For autonomous designs, add a wakeup source, kill switch, budget cap, no-progress stop, risk escalation boundary, and durable run ledger before allowing unattended mutation.
9. For scaffold work, inspect first, generate active unrestricted defaults, validate the config, and report exact files changed plus the command to run one manual iteration.

Bundled Scripts

Use these scripts when implementing or auditing a repo-local loop:

scripts/inspect_repo.py <target>              # detect instructions, scripts, CI, git state, .agent-loop
scripts/audit_loop.py <target>                # emit prioritized readiness findings
scripts/scaffold_loop.py <target> [--runner none|windows|github-actions|both] [--force]
scripts/generate_agents_md.py <target>        # create a compact starter AGENTS.md
scripts/validate_loop_config.py <target>      # fail on unsafe/missing autonomous-loop config
scripts/select_next_task.py <target>          # choose one ready low-risk task with verification
scripts/run_one.py <target> [--agent-command "..."] [--dry-run]
                                             # one locked/budgeted iteration; safe-stops by default
scripts/learn_from_ledger.py <target> [--write]
                                             # summarize recent runs and propose low-risk self-learning ratchets
scripts/append_ledger.py <target> --event safe_stop --reason kill_switch
scripts/summarize_run.py <target>             # update .agent-loop/handoff.md from ledger
scripts/report_loop.py <target> [html]        # generate .agent-loop/report.md or report.html
scripts/prompt_architecture_audit.py <target> # structural fable-class prompt/runtime architecture audit
scripts/prompt_profile.py <file>              # structural prompt profile; omits body text
scripts/ingest_prompt_architecture.py <source-file> <target> [--write]
                                             # compare an external prompt dump as untrusted data and write a sanitized source profile
scripts/generate_prompt_contract.py <target>  # create AGENT_RUNTIME_CONTRACT.md
scripts/build_agent_runtime_toolkit.py <target> [--runner none|windows|github-actions|both] [--source-prompt PATH] [--force]
                                             # scaffold a repo-local agent runtime toolkit with CLI, prompts, queue, state, and optional source-profile ingest
scripts/agent_loop.py <command> ...           # unified CLI: inspect/audit/scaffold/toolkit/validate/select/run-one/learn/report/prompt-audit/prompt-profile/ingest-prompt/generate-contract/evals
scripts/run_evals.py                          # smoke-test the bundled kit against fixtures

Scaffold defaults should be directly executable: status: active, kill_switch: false, optional verification, optional protected paths, and explicit config-based boundaries instead of hidden restrictions.

run_one.py must enforce safety in code, not only prose: lock acquisition, kill-switch safe-stop, clean-worktree preflight, ready low-risk task selection, wall-clock budget, optional agent command execution, checker command execution, protected-path diff/snapshot gate, ledger append, and restart artifacts such as .agent-loop/last-maker-prompt.md, .agent-loop/last-run.json, and .agent-loop/report.md.

Default Output

Use this shape unless the user asks otherwise:

[TARGET]
[MODE]
[CURRENT HARNESS / LOOP]
[GAPS]
[DESIGN]
[VERIFY]
[NEXT ACTION]

For MODE=scaffold, use this shape:

[TARGET]
[MODE=scaffold]
[INSPECTION]
[FILES CREATED / UPDATED]
[DEFAULT SAFETY POSTURE]
[VALIDATION]
[NEXT MANUAL RUN]

For MODE=audit, use this shape:

[TARGET]
[FINDING]
[SEVERITY]
[EVIDENCE]
[FIX]
[VERIFY COMMAND]
[DECISION]

Severity meanings:

• BLOCKER: the agent or loop can mutate real systems, spend unbounded budget, leak secrets, overwrite work, or declare success without independent verification.
• HIGH: the harness or loop is likely to fail on normal tasks because scope, state, environment, or verification is missing.
• MEDIUM: reliability will degrade over multiple runs; improve before scaling or delegating.
• INFO: useful hardening, documentation, or ergonomics; not blocking.

Startup Shortcuts

Explicit invocations:

$harness-loop-engineering MODE=harness TARGET=C:\path\to\repo FOCUS="create AGENTS.md and verification gates"
$harness-loop-engineering MODE=loop TARGET=C:\path\to\repo FOCUS="daily issue triage and safe PR loop"
$harness-loop-engineering MODE=autonomous TARGET=C:\path\to\repo FOCUS="self-learning scheduled agent loop with safe stop"
$harness-loop-engineering MODE=scaffold TARGET=C:\path\to\repo FOCUS="install unrestricted .agent-loop with Windows runner template"
$harness-loop-engineering MODE=toolkit TARGET=C:\path\to\repo FOCUS="build a repo-local agent runtime toolkit and operator CLI"
$harness-loop-engineering MODE=audit TARGET=C:\path\to\repo FOCUS="existing AGENTS.md, CI, state, subagents"

For a website, app, or product without a local repo, first identify the available evidence: source folder, deployed URL, docs, CI logs, browser traces, test reports, screenshots, or user-provided requirements. Do not invent a harness; report the missing source-of-truth surfaces and the smallest next evidence-gathering step.