release

# Release Commit Procedure When the user asks to prepare a release commit (e.g., "prep a commit for v217"), follow these steps in order. After a successful push, follow `references/github-release.md` for the GitHub release. ## Preflight: nothing outstanding on GitHub (blocking, before step 0) Run this from the release checkout (the repo root, on `dev`) before anything else, and again right before the push (`python3` on macOS): ``` python dev/release_preflight.py ``` - **BLOCKING:** open Dependabot, code-scanning or secret-scanning alerts; draft or triage security advisories (private vulnerability reports); open Dependabot PRs; a red latest push-CI run of any workflow on `main` or `dev`; a failed third-party check on either tip; real commits on `main` that the release checkout lacks (a hotfix never merged back: merge `origin/main` in and push). A check it cannot run also blocks, so a failed API call never reads as "nothing open". - **WARNING:** other open PRs, every open issue (tagged NEW since the last release), merged branches left on the remote. - **Exit 1 = STOP.** Show the user every item. Each blocker gets resolved (a Dependabot PR merged only after its platform e2e run is green, or closed; alerts fixed or dismissed with a reason; hotfixes back-merged) or the user explicitly accepts it. Only then re-run with `--ack "<key>"`, and only for keys the user named. Never ack on your own judgment. - **Report the warnings in the same message.** List any acked items in the release commit body so the decision is on record. Why: on 2026-09-11 embody.tools sign-in was down ~23h after a dependency bump (PR #105) and specimen submit had been broken 4 days by a rename; two hotfixes then sat on `main` without `dev`. The preflight keeps outstanding GitHub state from being skipped; the platform e2e gate is what catches a breaking bump. ## 0. Save the Project The entire save call is `project.save()` -- no arguments. TD increments the `.toe` filename's trailing build, the `onProjectPreSave` hook in `dev/embody/execute_src_ctrl.py` bumps `par.Version`, deletes the prior release `.tox`, and exports the new one. Filename and `par.Version` stay in lock-step. The release export honors `pre_release`/`post_release` hook DATs placed directly under the Embody COMP (none exist today; they fire on EVERY `project.save`, always in LIVE mode -- the Embody comp is never copy-staged, so such hooks would mutate the live comp). On any export failure the manifest is NOT written and the prior version's stale manifest is removed: a pre_release abort or save failure leaves NO release `.tox` for the new version, while a post_release failure leaves the fresh `.tox` WITHOUT a manifest -- check the log and `release/` before pushing. The self-updater's rollback backup export passes `run_hooks=False`, so shipped hooks never fire in user projects during updates. Don't pass a path (TD increments from *your* path's build, desyncing by one). Don't pre-set `par.Version`. Don't call `ExportPortableTox` directly. **If externalized files changed on disk while TD was closed** (e.g. a landed worktree diff), verify the affected DATs re-synced into the live network BEFORE saving: table DATs load their file only on the post-launch refresh sweep, which can run AFTER an early save -- and the portable export captures LIVE DAT state, shipping stale content (observed v6.0.133: the exported `palette_catalog` was missing all 267 just-landed 33070 rows). Pulse `op.Embody.par.Refresh` and spot-check the changed DATs (row counts, code markers), then save. If you've already mis-saved: rename the off-by-one `.toe` on disk to match `par.Version`, then have the user close TD without saving and reopen. Do **not** save again -- the hook will delete the just-correct release `.tox`. ## 1. Audit All Changes - Run `git diff --stat` and `git diff HEAD --name-status` to identify every changed, added, and deleted file. - Read the diffs for all core source files (EmbodyExt.py, TDXNExt.py, EnvoyExt.py, etc.) to understand what was fixed/added. - Read diffs for new test files to understand coverage additions. - Read diffs for docs, schema, and rule/skill files. ## 1b. Docs Audit: every change in this release is documented The changelog says a release HAPPENED; `docs/` is what users read to USE the thing. A feature or a behavior change that ships with only a changelog bullet is undocumented -- and the drift is invisible, because nothing fails. Walk the release diff (step 1) and, for EVERY user-visible change, name the `docs/` page that now describes it. Not "does a page exist" -- does the page say the new truth. | Change in the diff | Docs obligation | |---|---| | New feature, parameter, MCP tool, or Convoy operation | A section on its owning page, reachable from `mkdocs.yml` nav. A genuinely new subsystem gets its own page. | | Changed behavior (defaults, gating, retries, timeouts, statuses) | Update the page that states the OLD behavior. Grep the old wording -- it is usually in more than one place (a parameter row, a concept page, a troubleshooting row). | | New user-visible string (status text, dialog, error) | Add or fix its Troubleshooting row so a user who searches the literal text lands on the fix. | | Removed / retired feature, par, or tool | Delete or amend every page that still promises it. A stale promise is worse than a missing page. | | Parameter help text edited in TD | `docs/embody/parameters.md` mirrors par help -- keep them identical. | | Internal refactor with no user-visible effect | Nothing. Say so explicitly in the audit rather than skipping the question. | Mechanics: - `git diff HEAD --name-status` -> for each source file, ask what a USER could observe. If the answer is "nothing", write that down; if it is anything else, the docs edit is part of THIS release, not the next one. - Grep before you write: `grep -rn "<old behavior phrase>" docs/` finds the copies that would otherwise go stale. Fix all of them. - New page -> add it to `mkdocs.yml` nav, or it ships invisible. - `mkdocs build --strict` must pass (broken internal links are errors). It proves the page BUILDS; step 7 is what makes it PUBLISHED. - Docs written in this pass are part of the release commit, so the changelog bullet and the page land together. Report the audit as a short list -- "changed X -> documented at `docs/...`", "changed Y -> internal only" -- so a skipped obligation is visible rather than implied. ## 2. Update Changelog Add a new entry at the top of `docs/changelog.md`: ```markdown ## v5.0.XXX One-line summary of the release themes. - **Feature/fix name.** What changed, in 1-2 sentences. - ... ``` ### Length is a hard cap (standing user directive) **One line of theme. 3-5 bullets, one to two lines each. Under ~160 words.** Nobody reads a wall of bullets, and an exhaustive one reads as AI slop. Twelve changes still get <= 5 bullets: group by theme, and give the leftovers one combined **Fixes.** bullet rather than a bullet each. The detail lives in the commit log and the diff. But **too short is also wrong** -- do not drop a user-facing feature to make the count. A shipped tool, a new install path or a fixed crash each earn their line; only internals get cut. A small release can be one bullet (`v6.0.250` in `docs/changelog.md` is the model); a big one uses all five. **Unreleased intermediate versions fold into the published entry.** `project.save()` bumps the version on every save, so most numbers never ship. Consolidate the whole tag-to-tag range (`git log <lasttag>..HEAD`) into the one entry -- and never leave a shipped feature out of it because it landed under a version that was superseded. Always cut: - incident stories, repro narratives, root-cause retelling - internal mechanisms -- protocol shapes, private names, fallback ladders - a superseded version, beyond one short parenthetical - test counts, beyond `+N tests` folded into a bullet Then COUNT. Over 5 bullets or ~160 words, cut before shipping -- never ship long and offer to trim after. ## 3. Update README.md - **Version badge + minimum-build statements are AUTOMATED**: `project.save()` (via `execute_src_ctrl.updateVersionDocs`) rewrites the README version badge from `par.Version` and the minimum-TD-build lines in README.md, docs/index.md, and CONTRIBUTING.md from the running `app.build` (the build we save with IS the support floor). Verify they match rather than editing by hand; the `test_version_sync` suite fails on any drift. - **Release history**: Add a one-line entry at the top of the "Recent releases" list. - **Test suite count**: Update the count if new test files were added (count `dev/embody/unit_tests/test_*.py`). ## 4. Verify Template Sync **When updating a rule or skill in `.claude/`, also update the corresponding template DAT in `dev/embody/Embody/templates/` if one exists.** This applies on every edit, not just at release time -- drift between source and template ships stale guidance to user projects. Templates in `dev/embody/Embody/templates/` must stay in sync with their `.claude/` counterparts: | `.claude/` file | Template file | |---|---| | `rules/td-python.md` | `templates/text_rule_td_python.md` | | `rules/parameters.md` | `templates/text_rule_parameters.md` | | `rules/mcp-safety.md` | `templates/text_rule_mcp_safety.md` | | `rules/network-layout.md` | `templates/text_rule_network_layout.md` | | `rules/td-connectivity.md` | `templates/text_rule_td_connectivity.md` | | `rules/multi-session.md` | `templates/text_rule_multi_session.md` | | `rules/worktree-td-safety.md` | `templates/text_rule_worktree_td_safety.md` | | `rules/performance.md` | `templates/text_rule_performance.md` | | `rules/tdxn-economy.md` | `templates/text_rule_tdxn_economy.md` | | `skills/td-api-reference/SKILL.md` | `templates/text_skill_td_api_reference.md` | | `skills/movie-export/SKILL.md` | `templates/text_skill_movie_export.md` | | `skills/parameter-design/SKILL.md` | `templates/text_skill_parameter_design.md` | | `skills/td-recovery/SKILL.md` | `templates/text_skill_td_recovery.md` | | `skills/multi-session-etiquette/SKILL.md` | `templates/text_skill_multi_session_etiquette.md` | | `skills/create-operator/SKILL.md` | `templates/text_skill_create_operator.md` | | `skills/debug-operator/SKILL.md` | `templates/text_skill_debug_operator.md` | | `skills/externalize-operator/SKILL.md` | `templates/text_skill_externalize.md` | | `skills/create-extension/SKILL.md` | `templates/text_skill_create_extension.md` | | `skills/manage-annotations/SKILL.md` | `templates/text_skill_manage_annotations.md` | | `skills/mcp-tools-reference/SKILL.md` | `templates/text_skill_mcp_tools_reference.md` | | `skills/pop-networks/SKILL.md` | `templates/text_skill_pop_networks.md` | | `skills/visual-aesthetics/SKILL.md` | `templates/text_skill_visual_aesthetics.md` | | `skills/brief/SKILL.md` | `templates/text_skill_brief.md` | | `skills/merge-divergent-tox/SKILL.md` | `templates/text_skill_merge_divergent_tox.md` | | `skills/glsl-shaders/SKILL.md` | `templates/text_skill_glsl_shaders.md` | | `skills/operator-gotchas/SKILL.md` | `templates/text_skill_operator_gotchas.md` | | `skills/testing/SKILL.md` | `templates/text_skill_testing.md` | | `skills/collab/SKILL.md` | `templates/text_skill_collab.md` | | `skills/td-api-reference/references/background-work.md` | `templates/text_skill_td_api_reference__background_work.md` | | `skills/td-api-reference/references/heavy-build-safety.md` | `templates/text_skill_td_api_reference__heavy_build_safety.md` | | `skills/parameter-design/references/state-lifetimes.md` | `templates/text_skill_parameter_design__state_lifetimes.md` | | `skills/mcp-tools-reference/references/coordination.md` | `templates/text_skill_mcp_tools_reference__coordination.md` | | `skills/visual-aesthetics/references/craft.md` | `templates/text_skill_visual_aesthetics__craft.md` | | `skills/visual-aesthetics/references/look-recipes.md` | `templates/text_skill_visual_aesthetics__look_recipes.md` | | `skills/merge-divergent-tox/references/procedure.md` | `templates/text_skill_merge_divergent_tox__procedure.md` | This table is the source of truth for what ships; keep it in sync with `_TEMPLATE_MAP_RULES` / `_TEMPLATE_MAP_SKILLS` in `EmbodyExt.py` (the actual shipping map). Template files that exist on disk but are NOT in that map (e.g. `text_rule_commit_push_checklist.md`, `text_rule_github_release.md`, `text_rule_refresh_after_commit.py`) are orphans -- do not add them here. Templates should be UTF-8 with LF line endings and no BOM. Each template carries an Embody/Envoy generated-by HTML comment, and otherwise must match its `.claude/` counterpart in content -- diff them (normalizing any legacy BOM + line endings) and fix any drift. Dev-only rules and skills (e.g. `.claude/rules/commit-push-checklist.md`, `.claude/rules/skill-prerequisites.md`, `.claude/rules/code-brevity.md`, `.claude/rules/multi-agent-review.md`, `.claude/rules/destructive-tests.md`, `.claude/rules/embody-code-conventions.md`, `.claude/rules/refresh-after-commit.md`, `.claude/skills/release/` (this skill, incl. `references/github-release.md`), `.claude/skills/agent-tests/`, `.claude/skills/add-mcp-tool/`, `.claude/skills/run-tests/`) live under `.claude/` for Embody developers only and are NOT shipped to user projects -- they have no template counterpart. The root `CLAUDE.md` and `dev/embody/Embody/templates/text_claude.md` serve different audiences and are maintained independently. ## 4b. Re-Vendor the Convoy Host App The Convoy host-app daemon exists TWICE on purpose: the source of truth in `dev/convoy/`, and a vendored copy at `dev/embody/Embody/convoy/host/` carried inside the `.tox` as text DATs so that installing works with no network access. If the vendored copy is stale, the release ships an installer that writes an OLD daemon -- and nothing about the running project looks wrong. ``` python dev/convoy/vendor_host_modules.py --check # exit 1 = drift python dev/convoy/vendor_host_modules.py # re-vendor, then SAVE ``` The vendored DATs are `syncfile=True`, so copying the file is the whole re-vendor -- TD reloads the DAT on its own. **The new content only reaches the shipped `.tox` on the next `project.save()`**, so re-vendor BEFORE step 0's save, never after. `--check` reports four drift classes; only the first is fixed by copying: | Class | Meaning | Fix | |---|---|---| | `STALE` | content differs | re-vendor (this script) | | `MISSING` | daemon module has NO vendored DAT | create a text DAT of that name in the `host` COMP and externalize it (tag `py`) -- **a file copy alone does not make it ship** | | `ORPHAN` | vendored file with no daemon source | `delete_op` the DAT (deleting only the file lets the next save re-create it) | | `ok` | current | -- | `test_convoy_host_vendor.py` asserts the same thing on the CI matrix, comparing newline-normalized text (Embody writes CRLF on Windows; `.gitattributes` stores LF, so the committed bytes are identical). Do not "fix" a CRLF diff by changing how Embody writes files -- that path is shared by every externalized DAT in the project. Note `convoy_install.HOST_MODULES` is a hardcoded manifest and has gone stale twice already. It is not the gate; the parity test is. If you add a daemon module, the test tells you what else to do. ## 5. Fresh-Install Smoke (before the release commit) Cold-open smoke of the DEV project is not enough: a fresh install runs a different path (the shipped `.tox` in a virgin project -- `init()` lifecycle, baked par values, no dev checkout, no externalized files). The v6.0.145 empty-Update-Status miss shipped precisely because this step was skipped. Run it after step 0's save (it reads the manifest that save wrote), from the repo root -- one command, same on Windows and macOS: ``` dev/.venv-tests/Scripts/python.exe dev/release_testing/smoke_run.py