| name | harness-engineering-audit |
| description | Audit a repository for OpenAI-style harness engineering and Codex/OMX readiness. Use when reviewing AGENTS.md, Codex config, OMX workflows, MCP, hooks, rules, skills, subagents, docs authority, validation commands, repo legibility, visual product spec packs, generated artifact policy, or production readiness for AI-driven development. |
| license | MIT |
| version | 0.2.1 |
Harness Engineering Audit
Use this skill to turn a repository into a production-ready, agent-legible engineering environment optimized for Codex, OMX, and long-running AI-assisted development.
This skill is audit-first, AGENTS-first, and low-risk auto-approved. It may generate reports and planning artifacts before edits, then low-risk recommendations from the audit are approved for follow-up execution without another user approval. Medium/high-risk changes still require explicit approval.
Core operating model
- Inspect the repo and collect evidence.
- Score the repo against the harness-engineering rubric.
- Produce a strict report with low/medium/high-risk recommendations.
- Generate AGENTS.md priority and OMX handoff artifacts.
- Auto-approve low-risk fixes, with root
AGENTS.md as P0 when it is missing, oversized, stale, or lacks docs/validation pointers.
- Classify lifecycle as
greenfield-bootstrap, brownfield-cleanup, or mature-audit.
- Use OMX planning/execution or explicit setup modes for auto-approved low-risk fixes; defer medium/high-risk changes until explicit approval.
The default report location is:
.codex/reports/harness-engineering-audit/
Each new run replaces the previous generated report directory at that path to avoid stale audit confusion. The included scripts only overwrite the known report directory or an explicitly provided output directory that is marked as a harness-engineering audit report directory.
When to use
Use this skill when a user asks to audit or improve:
- harness engineering
- agentic development readiness
- Codex readiness
- OMX workflow quality
AGENTS.md scope and quality.codex/config.toml- MCP configuration
- skills
- hooks and rules
- subagent/team workflow
- docs authority
- validation commands
- command registries
- generated artifact policy
- stack/tooling upgrade recommendations
- scaffolding/legacy entropy
- repo legibility for AI agents
- production-readiness for agent-first development
- docs-to-production app readiness
- visual product spec packs from generated PNG/SVG targets, UI packs, stories, token packs, and component contracts
- frontend parity and product usability harnesses
- runtime evidence / local product done gates
- prompt-injection and instruction-trust readiness
Authoritative principles
Treat OpenAI/Codex guidance as normative:
AGENTS.md should be a concise map, not an encyclopedia.- Repo-local docs should be the system of record.
- Validation should be executable and evidence-backed.
- Skills should encode repeatable workflows.
- Domain vocabulary should be discoverable and progressively disclosed, so agents use project language consistently without bloating hot-path instructions.
- Docs should be gardened as living knowledge: raw sources, synthesized docs, indexes, logs, and health checks need explicit boundaries and maintenance workflows.
- MCP should solve real external/context access problems.
- Hooks/rules should be deterministic, scoped, and safe.
- Subagents/team workflows should have clear delegation, path scope, and review boundaries.
- Generated artifacts and scaffolding require lifecycle policies.
- Generated visual targets are design-intent evidence, not executable specifications; every priority target needs an adjacent screen contract.
- Implied behavior must be classified as
explicit, inferred, or unresolved; unresolved and sensitive inferred behavior stays approval-gated.
- Production app readiness requires traceability from user stories to visual targets, screen contracts, routes, APIs, components, tests, screenshots, accessibility evidence, and production deltas.
Use Claude/Cursor/Karpathy/third-party resources only as comparison material, not as authority over Codex behavior.
The vocabulary/domain-language audit surface is informed by Matt Pocock's Dictionary of AI Coding as comparison material: shared terms, handoff artifacts, progressive disclosure, and the distinction between deterministic checks and review are useful harness concepts, but this skill should not copy that repo's prose.
The doc-gardening audit surface is informed by Andrej Karpathy's LLM Wiki gist as comparison material: persistent markdown knowledge bases, source/wiki/schema layering, ingest/query/lint workflows, indexes/logs, and wiki health checks are useful harness concepts, but this skill should not copy that gist's prose.
Required evidence to inspect
Always inventory these surfaces when present:
- root
AGENTS.md
- nested
AGENTS.md and AGENTS.override.md
.codex/config.toml.codex/hooks.json.codex/rules/**.codex/prompts/**.agents/skills/**.omx/**.github/workflows/**- root README and docs indexes
docs/**- canonical product docs such as
docs/product/**, docs/stories/**, docs/screens/**, docs/design/**, docs/architecture/**, docs/contracts/**, and docs/quality/**
- generated/imported UI targets such as
docs/screens/**/target*.png, docs/screens/**/target*.svg, docs/ui/targets/**, and equivalent Figma/design-export packs
- screen and behavior contracts such as
screen.contract.yaml, interactions.yaml, motion.yaml, data-requirements.yaml, route maps, state machines, acceptance matrices, and traceability matrices
- design token and component packs such as
tokens.json, docs/design/tokens/**, docs/design/components/**, Style Dictionary/Tokens Studio config, component recipes, and Storybook stories
- visual QA surfaces such as
.storybook/**, *.stories.*, playwright.config.*, screenshot baselines, visual-regression policy, accessibility policy, and viewport/theme/reduced-motion matrices
- glossary / terminology / domain-language docs such as
docs/GLOSSARY.md, docs/TERMINOLOGY.md, internal/CONTEXT.md, internal/domain.md, and dictionary/**
- ADR / decision-record surfaces such as
docs/adr/**, docs/ADR/**, and internal/adr/**
- doc gardening / wiki / knowledge-base surfaces such as raw sources, generated/synthesized docs,
index.md, log.md, ingest/query/lint workflows, and stale/contradiction/orphan/broken-link checks
- validation scripts and package scripts
- test/build/lint/typecheck/smoke commands
- generated artifact directories
- scaffold/legacy/stage/demo/preview markers
- cross-agent files:
CLAUDE.md, .claude/**, .cursor/**, .windsurf/**, .cline/**, .gemini/**, .github/copilot-instructions.md
Docs-to-production harness audit
The audit now includes a stack-neutral Docs-to-Production App Readiness family. It detects repository capabilities and evaluates whether the repo exposes the harness layers needed for Codex/OMX to build a locally usable product from docs and design targets:
- Visual Product Spec Pack
- Visual Target Contracts
- User Story & Acceptance Pack
- Frontend Parity System
- Design Token & Component Pack
- Component Workbench / Storybook Harness
- Visual Regression Gate
- Product Usability Contract
- Runtime Evidence Harness
- Coverage & Traceability Harness
- Local Provider Simulator Harness
- Brownfield State Inventory Harness
- State Machine Harness
- Negative Path Harness
- Role Walkthrough Harness
- Runtime Observability Harness
- Local Product Done Gate
- Repo-scoped Codex skills
- Instruction Trust / Prompt Injection Audit
For UI-led work, the skill applies these tenets:
- a generated screenshot/mockup is an input artifact, not the contract;
screen.contract.yaml or an equivalent artifact must state route, viewport, theme, visual state, interactions, data/API assumptions, motion, accessibility, and approval-gated inferences;- the repo should keep a canonical docs set for product intent, user stories, screens, design tokens/components, architecture, contracts, and quality gates;
- a traceability matrix should connect each priority story to visual targets, screen contracts, routes, components, APIs, fixtures, tests, and evidence;
- visual completion should be proven through a workbench/Storybook surface, E2E/runtime evidence, accessibility checks, and human-approved visual baselines rather than raw prompt assertions.
The audit writes pre-mortem and readiness artifacts under .codex/reports/harness-engineering-audit/, including premortem.md, failure-mode-register.yaml, harness-gap-matrix.yaml, docs-to-production-readiness.md, capability-adapter-report.yaml, instruction-trust-report.md, and reusable OMX prompt files under recommended-omx-prompts/.
product-harness-setup can create low-risk harness templates and repo-scoped skill templates for visual-target ingest, design-token/component contracts, frontend parity, product usability, runtime evidence, traceability, visual QA, and final review. It must not implement application features, enable live providers, mutate security-sensitive configuration, or use real data.
Run the audit script
Before running the audit in an interactive session, prompt the user for the audit level and proceed with their selection. Offer these choices:
| Choice | Side effects | Approval posture |
|---|
minimal (default) / audit | Report-only audit; no target source mutation. | Safe default. |
safe-setup | Create missing low-risk docs/templates only; never creates .codex/agents. | Explicit setup selection. |
force-ideal-harness | Stronger low-risk docs/template consolidation; no deletes or CI/config/hooks/security changes. | Explicit setup selection. |
symphony-repo-local | Repo-local Symphony contracts/templates and inert handoff text. | Explicit setup selection; live installs still approval-gated. |
symphony-live-handoff | Approval-gated handoff text in the report directory only; no target source or install/config mutation. | Explicit handoff selection. |
full-orchestration | Lane-pack orchestration contracts and harness custom-agent TOML; never runs agents or live install/config commands. | Explicit opt-in only; never the default. |
product-harness-setup | Docs-to-production harness templates, local-product-done docs, and repo-scoped Codex skill templates; no app implementation. | Explicit setup selection. |
The Python script also prompts for this level when run from an interactive TTY without --mode. Non-interactive runs default to minimal/audit so automation remains stable.
From the repo root after installing this skill, run the script from the installed location.
Project-scoped install:
python3 .agents/skills/harness-engineering-audit/scripts/run_audit.py .
User-scoped install:
python3 ~/.codex/skills/harness-engineering-audit/scripts/run_audit.py .
The script writes:
.codex/reports/harness-engineering-audit/
inventory.json
scorecard.json
stack-inventory.json
tool-inventory.json
upgrade-recommendations.json
upgrade-recommendations.md
web-verification-queue.json
source-trust-policy.md
update-status.json
report.md
findings.md
recommended-fixes.md
agents-priority.md
omx-handoff.md
next-step.md
next-step.json
docs-to-production-readiness.md
premortem.md
premortem.json
failure-mode-register.yaml
harness-gap-matrix.yaml
capability-adapter-report.yaml
instruction-trust-report.md
recommended-omx-prompts/
prompts/
deep-interview.md
ralplan.md
team.md
ralph.md
symphony-adoption.md
tool-upgrade-ralplan.md
Recommendation defaults are intentionally conservative:
- recommend tools: true
- web verification requested: true
- human approval gate: required
- install/config mutation: false
- write reports: true
Local Python scripts cannot browse. They therefore write a web verification queue and keep web_verified: false until a browsing-capable human/agent records evidence. Generated install/config/validation/rollback commands are inert handoff text and must not be executed without explicit approval.
Primary artifacts now include a “What happened / What did not happen / What needs approval” section plus machine-readable mode_summary, approval_state, tool_recommendation_state, and current_step_explanation fields in next-step.json.
Use a custom output directory only when needed:
python3 ~/.codex/skills/harness-engineering-audit/scripts/run_audit.py . --out /tmp/harness-engineering-audit
Execution modes
The default prompt choice remains report-only. Use minimal or audit explicitly when you want to skip the prompt while preserving report-only behavior:
python3 ~/.codex/skills/harness-engineering-audit/scripts/run_audit.py . --mode minimal
python3 ~/.codex/skills/harness-engineering-audit/scripts/run_audit.py . --mode audit
Additional explicit modes create only low-risk harness artifacts with provenance markers and setup-rollback-manifest.json:
AUDIT_SCRIPT=~/.codex/skills/harness-engineering-audit/scripts/run_audit.py
python3 "$AUDIT_SCRIPT" . --mode safe-setup
python3 "$AUDIT_SCRIPT" . --mode force-ideal-harness
python3 "$AUDIT_SCRIPT" . --mode symphony-repo-local
python3 "$AUDIT_SCRIPT" . --mode symphony-live-handoff
python3 "$AUDIT_SCRIPT" . --mode full-orchestration
python3 "$AUDIT_SCRIPT" . --mode product-harness-setup
--force keeps its existing output-overwrite meaning; it is not the force-ideal-harness mode.
Mode boundaries:
audit: reports only, no target source mutation.safe-setup: creates missing low-risk docs/templates only.force-ideal-harness: explicit stronger low-risk consolidation; no deletes or CI/config/hooks/security changes.symphony-repo-local: repo-local Symphony contracts/templates and inert handoff text only.symphony-live-handoff: approval-gated handoff text only; no install/config mutation.full-orchestration: explicit opt-in for lane-pack orchestration contracts and project custom-agent TOML. It is never the default, never runs agents, and never executes live install/config commands.product-harness-setup: explicit opt-in for docs-to-production harness docs and repo-scoped skill templates, including visual product spec pack templates; it never implements app code or enables live providers.
Lane-pack boundaries:
safe-setup creates lane-pack Markdown docs under docs/harness/** only and must not create .codex/agents.- Stack-detected lane packs must report activation confidence, matched evidence, and recommendation policy. Weak signals remain advisory candidates instead of default missing-lane work.
.codex/agents/*.toml custom-agent definitions are generated only by full-orchestration, use harness-* names, and are invoked only when the user explicitly asks Codex to use subagents.- The lane-pack registry uses universal core lanes plus stack-detected lanes such as UI/UX, backend/API, data, security, performance, infra, AI/ML/CV, docs gardening, and QA.
Skill update behavior
Normal audit runs check this skill's update status by default and write update-status.json plus a Skill update status section in report.md. This check is report-only and never updates skill files. Disable it with:
python3 ~/.codex/skills/harness-engineering-audit/scripts/run_audit.py . --no-check-update
Self-update is explicit and approval-gated. To update this one user-scoped skill installation, run:
python3 ~/.codex/skills/harness-engineering-audit/scripts/run_audit.py . --self-update --update-scope user
The self-update path runs only this command and exits immediately after success:
gh skill install ryne2010/harness-engineering-audit skills/harness-engineering-audit --agent codex --scope user --force
Project-scoped installs should generally be updated intentionally through the repository and reviewed in a PR. Avoid gh skill update --all for this flow because system or manually installed skills may lack GitHub metadata and because this skill should update only itself.
Score dimensions
The report scores:
- Agent Legibility
- Instruction Hygiene
- Docs Authority
- Vocabulary / Domain Language Control
- Doc Gardening / Knowledge Base Readiness
- Validation Truth
- Harness Feedback Loops
- Codex Config Readiness
- Skills Readiness
- MCP Readiness
- Hooks / Rules Safety
- Subagent / OMX Workflow
- Cross-Agent Compatibility
- Entropy / Scaffolding Control
- Production Readiness
- Symphony Orchestration Readiness
- Docs-to-Production App Readiness
Scores are strict and intentionally opinionated. A score is not a final truth; it is an evidence-backed signal to guide planning.
The scorecard also includes a lifecycle object and a readiness registry:
score_schema_version: "2"lifecycle.classificationreadiness_registry.categories
The readiness registry exposes additional harness/Symphony categories without silently adding every category as a top-level score dimension.
Fix classification
Classify every recommendation by confidence and risk.
Low-risk / high-confidence
Auto-approved by default for an OMX follow-up pass. AGENTS.md items are P0 and should be handled first:
- trim duplicated root
AGENTS.md content
- add or refresh concise root
AGENTS.md with docs and validation pointers
- preserve or clarify nested
AGENTS.md scope boundaries
- add missing docs index links
- document real validation commands
- add a command registry
- add a validation matrix
- add or link concise glossary/domain-language guidance
- add a doc gardening workflow for indexes, logs, source boundaries, and docs health checks
- clarify deterministic automated checks versus judgment-based automated/human review
- add a generated artifact policy
- add a DX preflight script
- classify generated artifacts
- add a repo operating model doc
- document existing MCP/skills/hooks without changing behavior
Medium-risk
Requires explicit approval before execution:
- restructure docs folders
- add or change MCP servers
- change hook behavior
- change CI validation behavior
- move generated artifacts
- add new skills
- create new repo-level automation
High-risk
Do not execute without explicit approval:
- delete scaffolding or legacy paths
- rewrite CI or deployment workflows
- change package scripts used by production
- remove hooks/rules/MCP servers used by teams
- alter subagent/team workflow semantics
- change security or sandbox settings
Low-typing OMX continuation
Do not make the user copy or retype the long suggested prompts.
After the audit script runs:
- Read
report.md enough to present the summary required below.
- Use
next-step.json / next-step.md to identify the default next stage.
- In an interactive OMX runtime, present a structured selection:
- the generated default next stage from
next-step.json (default/recommended)
- Execute auto-approved fixes (only if an execution-ready plan already exists)
- Review risky questions (only for medium/high-risk ambiguity)
- Stop after audit
- If the user selects a stage, continue in the same session with the corresponding generated prompt file in
prompts/; do not ask them to paste the command.
- If no structured selection UI is available, print only the short resume command:
$harness-engineering-audit continue.
When the user runs $harness-engineering-audit continue, inspect .codex/reports/harness-engineering-audit/next-step.json. If it exists, resume from its default/recommended stage or present the same selection. If it does not exist, run the audit first.
AGENTS.md priority lane
Treat AGENTS.md improvements as first-class harness fixes:
- Root
AGENTS.md is P0 when missing, larger than the concise hot-path target, missing validation commands, missing docs/source-of-truth pointers, or missing nested instruction scope guidance.
- Low-risk AGENTS.md changes are auto-approved. Do not wait for another approval to add, trim, or refresh root
AGENTS.md as a concise operating map.
- Keep root
AGENTS.md map-like. Move detailed process prose into docs and link to it.
- Preserve nested AGENTS semantics. Do not delete nested instruction files without explicit approval.
- Do not change hooks, MCP, CI, package scripts, security/sandbox settings, source structure, or delete scaffolding as part of the auto-approved AGENTS.md lane.
OMX handoff workflow
After generating omx-handoff.md, the generated prompt files encode this flow:
$ralplan "Read .codex/reports/harness-engineering-audit/prompts/ralplan.md and follow it."
$team "Read .codex/reports/harness-engineering-audit/prompts/team.md and follow it."
$ralph "Read .codex/reports/harness-engineering-audit/prompts/ralph.md and follow it."
$deep-interview "Read .codex/reports/harness-engineering-audit/prompts/deep-interview.md and follow it." # only for medium/high-risk questions
Output discipline
When presenting results, include:
- overall verdict
- score summary
- strongest surfaces
- weakest surfaces
- auto-approved low-risk fixes, especially AGENTS.md P0 items
- medium/high-risk deferred recommendations
- report path
- next OMX selection/resume command
Do not claim the repo is production-ready unless validation truth, instruction hygiene, docs authority, and feedback loops are all strong and evidenced.
