datadog-rules-go-orchestrion-upstream-migration

<!-- Unless explicitly stated otherwise all files in this repository are licensed under the Apache 2.0 License. This product includes software developed at Datadog (https://www.datadoghq.com/) Copyright 2025-Present Datadog, Inc. --> # Datadog rules_go Orchestrion upstream migration Use this skill when you need to move the vendored Orchestrion-enabled `rules_go` fork in this repository from the currently pinned upstream `rules_go` base to another upstream tag or commit. This skill is intentionally repository-local: it is stored as a Codex-compatible skill, but any agent can read it as a normal implementation guide. The manifest-driven Go/Python sync API does not change the fork migration boundary. A migration must preserve disabled empty-repository behavior, Bazel-managed Go SDK bootstrap, and the existing public Orchestrion aliases; it must not add target discovery, service naming, or manifest policy to `rules_go`. ## Non-negotiable contract Preserve the maintained fork contract: - Migrate the target upstream's `base` tree first. - Treat `base` as the clean, generic Orchestrion integration. - Maintain both the patch series under `third_party/rules_go_orchestrion/patches/<upstream>/` and the materialized trees selected by `third_party/rules_go_orchestrion/registry.json`. - Maintain public consumer patch profiles under `third_party/rules_go_orchestrion/profiles/`. - Do not reintroduce a public `complete` variant or public consumer-specific patch bundles. - Do not edit generated `*.CHANGED_FILES.md` reports by hand. - Do not update `*.METADATA.json` to a new upstream tag or commit until the vendored tree actually reflects that upstream base plus the Datadog delta. - Do not call the migration complete until materialization checks, profile verification, changed-file regeneration, and relevant smoke or integration validation have run. - Preserve deterministic action outputs for `GoStdlib`, `GoSyntheticTestmainHelpers`, synthetic `GoCompilePkg` (`~testmain.a`), and `GoLink`. An ordinary `GoStdlib` action must not publish or consume the instrumented stdlib cache. A Test Optimization build must publish stable woven archives across two isolated output roots, including CGO mode. ## First actions 1. Confirm the requested target upstream: - exact upstream `rules_go` tag or commit - whether the work is exploratory or intended for a pull request - whether the new upstream must be included in public consumer patch profile generation immediately 2. Inspect the current repository state before editing: - `git status --short` - `third_party/rules_go_orchestrion/registry.json` - the target upstream's `*.METADATA.json` - the target upstream's `*.CHANGED_FILES.md` - `third_party/rules_go_orchestrion/profiles/workspace_runtime.json` 3. Read the maintainer docs that define the current fork model: - `docs/rules_go_orchestrion_support_selection.md` - `docs/rules_go_variant_maintenance_guide.md` - `docs/go_orchestrion_maintainer_state.md` - `docs/go_orchestrion_bazel_deep_dive.md` - `CONTRIBUTING.md` 4. Inspect the current patch/tree relationship before changing the fork: - `python3 tools/dev/generate_rules_go_fork_maps.py --check` - `python3 tools/dev/materialize_rules_go_fork.py check --all` - `python3 tools/dev/verify_rules_go_profiles.py --public-denylist tools/dev/private_leak_public_denylist.txt` 5. Install `zstd` before running the local generated-profile verifier; it reads Bazel's compact execution logs. ## Implementation path - **Migration workflow:** follow [migration-workflow.md](references/migration-workflow.md). - **Validation:** follow [validation-checklist.md](references/validation-checklist.md). - **Troubleshooting:** follow [troubleshooting.md](references/troubleshooting.md). ## Sensitive surfaces Expect conflicts or semantic drift around these paths first: - `go/orchestrion_workspace.bzl` - `go/private/orchestrion/*` - `go/private/context.bzl` - `go/private/actions/compilepkg.bzl` - `go/private/actions/stdlib.bzl` - `go/private/actions/link.bzl` - `go/private/actions/archive.bzl` - `go/private/rules/library.bzl` - `go/private/rules/test.bzl` - `go/private/rules/stdlib.bzl` - `go/private/rules/transition.bzl` - `go/tools/builders/builder.go` - `go/tools/builders/compilepkg.go` - `go/tools/builders/env.go` - `go/tools/builders/env_orchestrion.go` - `go/tools/builders/importcfg.go` - `go/tools/builders/link.go` - `go/tools/builders/module_proxy.go` - `go/tools/builders/orchestrion*.go` - `go/tools/builders/stdlib.go` - `go/tools/builders/tool_version.go` Pay special attention to: - Orchestrion `toolexec` wiring - synthetic `testmain` handling - stdlib weaving and stdlib archive/cache persistence - offline module proxy inputs - `dd_trace_go_versions.json` - tool-version validation - Bzlmod and WORKSPACE extension entrypoints - Bazel transitions that must preserve Orchestrion settings ## Done criteria A migration is done only when all of these are true: - `base` points at the requested upstream base plus the Datadog Orchestrion integration. - The target upstream's `*.METADATA.json` file names the upstream repository, commit, and tag when a tag exists. - The target upstream's `*.CHANGED_FILES.md` report was regenerated by `tools/dev/diff_rules_go_fork.py`. - `python3 tools/dev/materialize_rules_go_fork.py check --all` passes. - `python3 tools/dev/verify_rules_go_profiles.py --public-denylist tools/dev/private_leak_public_denylist.txt` passes. - Profile verification executes two isolated builds for both ordinary and Test Optimization modes and compares the four required action families. The ordinary declared stdlib cache is empty; the instrumented CGO stdlib contains Test Optimization markers and produces identical bytes. - The selected validation lanes in [validation-checklist.md](references/validation-checklist.md) pass, or every skipped lane has a concrete reason. - Any fixture or consumer validation that was required for the changed surface has passed. - The final report names the target upstream, changed-path counts, validation results, and any remaining external blockers. ## Test Optimization alias contract When validating a consumer that uses Test Optimization, preserve the stable alias contract from the vendored base tree: ```bazelrc common:test-optimization --repo_env=DD_TEST_OPTIMIZATION_ENABLED=1 ``` `--config=test-optimization` is the only user-facing metadata switch. Optimized targets enable Orchestrion through their transition. Omitting the config must leave metadata bootstrap disabled for public Go extension repositories and for low-level repositories explicitly configured with `enabled_by_env = True`. ## Stop conditions Stop and escalate instead of guessing when: - The requested target upstream tag or commit is ambiguous. - Upstream `rules_go` changed a build pipeline surface enough that the Orchestrion behavior cannot be mapped without a design decision. - `base` needs behavior that appears specific to one Datadog monorepo consumer. - The patch series applies but the materialized tree check differs from the checked-in tree and the difference is not intentionally regenerated. - Smoke validation passes but runtime validation shows no CI Visibility startup or no payload files. - The only available fix requires a public consumer-specific patch bundle. - Required validation needs credentials or environment state that is not available locally.