| name | approval-eval-cache-lifecycle |
| description | Use this skill when changing draft approval, protected artifact metadata, eval replay, verification thresholds, repo-local cache refresh, export policy, or explain output. |
Approval Eval Cache Lifecycle
Purpose
Maintain the trust lifecycle after a draft pack exists: human review, approval or rejection, local eval replay, verified status decisions, pinned repo-local caches, managed exports, and explain output.
When To Use
Use this for approval.py, eval_replay.py, cache_manager.py, explain.py, status transitions, approval metadata, eval scoring, cache metadata, or export rules.
Inspect First
src/orgs_ai_harness/approval.py for review views, exclusions, protected artifacts, and approval traces.src/orgs_ai_harness/eval_replay.py for adapter contract, scoring, thresholds, traces, and report updates.src/orgs_ai_harness/cache_manager.py for cache refresh and export policy.src/orgs_ai_harness/explain.py for displayed state and boundary decisions.local-docs/development-phases/sprint-06-review-approval-and-protected-artifacts.md, sprint-07-local-eval-replay-and-verification.md, and the full application guide.
Repository Map
- Draft artifacts live in
org-agent-skills/repos/<repo-id>/.
- Approval metadata:
approval.yml.
- Eval report:
eval-report.yml; eval traces append to trace-summaries/eval-events.jsonl.
- Repo-local cache:
<covered-repo>/.agent-harness/cache/.
- Runtime exports:
<covered-repo>/.agent-harness/cache/exports/<target>/.
Standard Workflow
- Review draft artifacts with
approve <repo-id> before accepting anything.
- Approval writes protected artifact hashes and transitions to
approved-unverified.
- Rejection preserves draft artifacts and records a trace.
- Eval refuses drafts unless
--development is passed.
- Verification requires approved artifacts and approved evals, then applies thresholds.
- Cache refresh is allowed only for
approved-unverified or verified packs.
- Export reads from the pinned cache and enforces draft/development policy flags.
- Explain should show coverage, status, cache, approved skills, evals, unknowns, proposals, and boundary decisions.
Thresholds
verified requires no blocking unknowns or safety failures and either at least 0.20 pass-rate improvement or at least 0.30 rediscovery-cost reduction.- Otherwise approved packs remain
approved-unverified unless failures force needs-investigation.
Validation Commands
PYTHONPATH="$PWD/src" python3 -m unittest tests.test_org_pack_foundation.RepoOnboardingTests
PYTHONPATH="$PWD/src" python3 -m orgs_ai_harness validate <repo-id>
Use concrete CLI smoke flows in temp fixtures for lifecycle changes:
PYTHONPATH="$PWD/src" python3 -m orgs_ai_harness approve <repo-id> --all
PYTHONPATH="$PWD/src" python3 -m orgs_ai_harness eval <repo-id>
PYTHONPATH="$PWD/src" python3 -m orgs_ai_harness cache refresh <repo-id>
PYTHONPATH="$PWD/src" python3 -m orgs_ai_harness export codex <repo-id>
Invariants
- Approval is explicit; draft packs are not trusted by generation alone.
- Protected artifact hashes must match approved artifacts exactly.
- Development evals and development exports must not imply verification.
- Cache metadata records pack status, pack ref, source pack ref, org skill pack path, warnings, and applied proposals.
- Cache and exports are made read-only after refresh/export.
Common Pitfalls
- Allowing onboarding regeneration to overwrite approved artifacts.
- Marking a pack verified despite open blocking unknowns.
- Exporting stale cache after applied proposals.
- Forgetting approved-unverified warning metadata.
- Changing report fields without updating explain and tests.
Escalation
If introducing a new adapter or export target, keep the fixture adapter deterministic and add policy checks before enabling broader execution.
