Generates a bash script that bulk-mirrors NVCF images (and optionally helm charts) from NGC to Azure Container Registry by iterating an `image-manifest/manifest.yaml` SBOM. Resolves the NGC source path **per manifest group** from a built-in mapping table that mirrors `image-manifest/source-registry-config.yaml` (see the per-group source mapping below), overridable per-group via `streaming-env.sh`. Two emit modes — `regsync` (declarative, idempotent, parallel; preferred) or `az acr import` loop (no extra deps). Handles scalar and list tag values, skips `helm-*` keys for the image pass, and applies the `<acr>.azurecr.io/<sub-path>/<image>:<tag>` convention so NVCF helm charts (which concatenate `<registry>/<repository>/<image>`) don't render double-slash paths. Use after the `acr` foundation skill has provisioned the ACR and before `nvcf-self-managed-installation`. Trigger when the user mentions "mirror images", "populate ACR", "regsync", "az acr import", "NGC → ACR", "bulk mirror", "image-manifest sync", or "s
Instalación
Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.
Generates a bash script that bulk-mirrors NVCF images (and optionally helm charts) from NGC to Azure Container Registry by iterating an `image-manifest/manifest.yaml` SBOM. Resolves the NGC source path **per manifest group** from a built-in mapping table that mirrors `image-manifest/source-registry-config.yaml` (see the per-group source mapping below), overridable per-group via `streaming-env.sh`. Two emit modes — `regsync` (declarative, idempotent, parallel; preferred) or `az acr import` loop (no extra deps). Handles scalar and list tag values, skips `helm-*` keys for the image pass, and applies the `<acr>.azurecr.io/<sub-path>/<image>:<tag>` convention so NVCF helm charts (which concatenate `<registry>/<repository>/<image>`) don't render double-slash paths. Use after the `acr` foundation skill has provisioned the ACR and before `nvcf-self-managed-installation`. Trigger when the user mentions "mirror images", "populate ACR", "regsync", "az acr import", "NGC → ACR", "bulk mirror", "image-manifest sync", or "sync NVCF SBOM to ACR".
Bulk mirror NGC → ACR
What this skill produces
A self-contained bash script that:
Locates the NVCF image-manifest SBOM (image-manifest/manifest.yaml at the repo root of a streaming-staging clone) — see Manifest sources for how this works from a streaming-staging clone vs an azure-base-only checkout.
Iterates the nvcf-self-managed-stack group, emitting one mirror entry per image:tag pair.
Skips chart artifacts (keys prefixed helm-) for the image pass. Mirroring them is covered in Helm chart mirroring.
Handles both scalar values (bitnami-cassandra: 5.0.6-nv-1) and list values (nvcf-openbao: [2.2.2-nv-1, 2.5.1-nv-1.1.0] during upgrade transitions).
Writes either a regsync.yaml + run command (default), or a loop of az acr import commands.
This skill is the operational counterpart to acr: acr creates the registry; acr-mirror populates it.
When to use it
In the aks-bootstrap flow, this is stage 04b — runs after acr (stage 04) creates the registry and before nvcf-self-managed-installation (post-foundation) installs the helmfile bundle, because that install tries to pull images from <acr>.azurecr.io/<repo>/....
It is not part of the mandatory aks-bootstrap cycle — bootstrap is foundation-only. Mirroring is the first post-foundation step.
Inputs
Sourced from streaming-env.sh and the upstream stages' generated/.env.* files.
Required:
Variable
Description
Example
ACR_NAME
Target ACR (from streaming-env.sh)
acrnvcftestwestus3
NGC_API_KEY_FILE
Path to a mode-600 file containing the raw NGC API key
Used only when FETCH_FROM_GITLAB=true — tag or SHA of the manifest commit to fetch
main
FETCH_FROM_GITLAB
If true and MANIFEST_PATH doesn't exist, fetch the manifest from omniverse/ovonsm/image-manifest via the GitLab API. Requires GITLAB_TOKEN.
false
NGC_SOURCE_REGISTRY
Source registry host
nvcr.io
TARGET_REPOSITORY
Sub-path under the ACR. Never empty — see Why the sub-path matters. Defaults to ${CLUSTER_NAME} so multiple deployments sharing one ACR don't collide.
${CLUSTER_NAME}
INCLUDE_GROUPS
Comma-separated manifest groups to mirror (subset filter). Empty = all non-metadata groups.
(all)
NGC_SOURCE_REPO_<GROUP>
Per-group override for the NGC source path. Group hyphens become underscores in the env var name (e.g. NGC_SOURCE_REPO_nvcf_self_managed_stack). See Per-group source mapping for built-in defaults.
(defaults below)
MIRROR_TOOL
regsync (recommended) or az_acr_import
regsync
INCLUDE_HELM_CHARTS
Also mirror helm-* chart artifacts (OCI helm)
false
SKIP_KEYS
Comma-separated manifest keys to skip by name — entries that are not OCI artifacts and 404 if mirrored. The skill can't read component_type at runtime (versions.yaml, which carries it, isn't shipped beside manifest.yaml), so this is a hand-maintained list. Default nvcf-stack is component_type: package upstream (an SBOM umbrella label).
nvcf-stack
DRY_RUN
Skip the actual push; emit the config and print what would be mirrored
false
Per-group source mapping
The skill ships with these defaults for every group currently in manifest.yaml:
Manifest group
Default NGC source path (under nvcr.io/)
nvcf-self-managed-stack
0651155215864979/ncp-dev
nvcf-compute-plane-stack
0651155215864979/ncp-dev
container_cache
0651155215864979/ncp-dev
gxcache
0833294136851237/nvcf-ncp-staging
lls
0795883928596950
ucc
nvidia/omniverse
ddcs
nvidia/omniverse
storage-api
nvidia/omniverse
These mirror image-manifest/source-registry-config.yaml (the deployment source of truth) — group keys match manifest.yaml exactly (container_cache, not caches). A group present in the manifest but absent from this map is a hard error, not a silent skip; use INCLUDE_GROUPS to scope a partial run.
To override per-group: set NGC_SOURCE_REPO_<GROUP> in streaming-env.sh, replacing hyphens with underscores. Example:
# In streaming-env.shexport NGC_SOURCE_REPO_nvcf_self_managed_stack="0833294136851237/some-other-team"export NGC_SOURCE_REPO_storage_api="nvidia/something-else"
A group that's in the manifest but missing from both the built-in defaults AND any override produces a WARN and is skipped — the script doesn't fail, so unknown future groups don't break existing mirrors. Set the env var to include them.
Multiple NGC source orgs — one key may not cover them all
The groups source from more than one NGC org (see the per-group source table above for the current paths). The mirror authenticates to nvcr.io with a single NGC_API_KEY, and access to private orgs is membership-scoped: if the key's NGC user isn't a member of a private source org, the pull returns 403 Forbidden for that org — which looks like a bad or expired key but is actually an org-access gap, and is easy to lose hours to.
Public orgs don't gate on membership. Some groups source from a public NGC org, where any valid NGC key resolves the images — it's the authenticated token that matters, not org membership. Only the private source orgs require membership, so a key that belongs to the private org you source the control-plane groups from will also pull the public-org groups.
Consequences:
Usually one key is enough: a key whose user is a member of the private org(s) the manifest sources from covers everything, public-org groups included. Point NGC_API_KEY_FILE at it.
Two private orgs need per-org passes: the multi-key problem only bites when the manifest spans two private orgs that no single key is a member of both (e.g. the key belongs to one private org but a group is sourced from a different private org — the failure in the bug). Then run the mirror once per private org — INCLUDE_GROUPS to that org's groups, NGC_API_KEY_FILE to that org's key:
# Pass 1 — groups sourced from private org A, using org A's key
INCLUDE_GROUPS="<groups-for-org-A>" NGC_API_KEY_FILE=~/.nvcf-secrets/ngc-key-A ./<script>.sh
# Pass 2 — groups sourced from private org B, using org B's key
INCLUDE_GROUPS="<groups-for-org-B>" NGC_API_KEY_FILE=~/.nvcf-secrets/ngc-key-B ./<script>.sh
# (Public-org groups can ride along in any pass — any valid key pulls them.)
This applies only to the mirror step (which pulls from NGC). Once everything lands in the one ACR, the split disappears for the cluster: AcrPull node identity authenticates all pulls, so no per-org pull secrets are needed in-cluster.
Outputs
Bash:generated/stageNN-<cluster-name>-acr-mirror.sh (NN = next available stage number in the deployment cycle; commonly 04b or 08).
regsync config (when MIRROR_TOOL=regsync):generated/regsync.yaml — emitted at script runtime; consumed by regsync once -c generated/regsync.yaml.
The script does NOT write a .env.acr-mirror file because nothing downstream consumes its outputs — successful exit is the signal. It does print a final tally line (mirrored N images, M charts) and (in DRY_RUN=true) the list of pairs without executing.
Manifest sources
Two scenarios, in priority order:
From a streaming-staging clone
image-manifest/manifest.yaml ships at the repo root, pre-synced from omniverse/ovonsm/image-manifest (alongside a COMMIT file for provenance and source-registry-config.yaml). The default MANIFEST_PATH=${REPO_ROOT}/image-manifest/manifest.yaml resolves correctly. To advance to a newer manifest, pin a different streaming-staging tag/SHA and git pull.
From an azure-base-only checkout
No manifest ships with azure-base. Two options:
Set MANIFEST_PATH in streaming-env.sh to a local manifest checkout (e.g., a sibling clone of omniverse/ovonsm/image-manifest).
Set FETCH_FROM_GITLAB=true and provide GITLAB_TOKEN. The script fetches the manifest from the GitLab API at MANIFEST_REF — equivalent to what streaming-staging's sync-nvcf.sh does, but at deploy time.
The script echoes manifest source: <path> and the version / generated_at fields so the source is auditable from logs.
Why the sub-path matters
Two reasons the sub-path cannot be empty or static:
Empty breaks helm pulls. NVCF helm charts concatenate <registry>/<repository>/<image>:<tag>. The helmfile env file's global.image.repository is set to match TARGET_REPOSITORY. If TARGET_REPOSITORY is empty, the chart renders <acr>.azurecr.io//<image>:<tag> (double slash), which docker pull and many container runtimes reject.
A static sub-path collides across deployments. When two clusters share one ACR (team registry, prod ACR, dev/staging side-by-side), both mirroring to <acr>/nvcf/<image>:<tag> race to overwrite tags — last write wins, and if the two manifest commits pin different versions of the same image, one cluster pulls the wrong tag.
The default ${CLUSTER_NAME} gives every deployment its own namespace under the ACR. After mirror, set in the helmfile env file:
global:image:registry:<acr>.azurecr.io# not nvcr.iorepository:aks-nvcf-validation-westus3# matches TARGET_REPOSITORY (= ${CLUSTER_NAME})helm:sources:registry:<acr>.azurecr.iorepository:aks-nvcf-validation-westus3
No image-pull secret needed once you point at the ACR. The acr skill granted the AKS kubelet identity AcrPull on this registry, so a pull from <acr>.azurecr.io/... is authenticated by the node identity. Once global.image and global.helm.sources point at the ACR you can drop the entire NGC pull-secret apparatus the NGC-direct path requires: the per-namespace nvcr-pull-secret, the global.imagePullSecrets entry, and the docker login/helm registry login nvcr.io steps. This also removes the multi-org pull-secret problem (function pods otherwise pull several nvcr.io orgs, each needing a different key) — everything is one AcrPull-authenticated registry. See the "Pull from a registry mirror (ACR/ECR)" section of the nvcf-self-managed-installation skill for the consumer-side env + step changes.
Override TARGET_REPOSITORY only when the ACR has an existing namespacing convention (e.g., prod/<release-name>) or when multiple deployments must share tags.
Manifest schema (consumed by this skill)
version:'1.0'generated_at:'2026-05-28T22:34:11Z'nvcf-self-managed-stack:bitnami-cassandra:5.0.6-nv-1# scalarnvcf-openbao:# list (upgrade transition; rare)-2.2.2-nv-1-2.5.1-nv-1.1.0helm-nvcf-cassandra:0.11.1# chart artifact — skipped in image passgxcache:gxcache-init:1e47f722gxcache-service:b206ce39helm-gxcache:0.8.2lls:gdn-streaming:2.0.1streaming-proxy:2.0.1storage-api:helm-discovery-service:2.3.8storage-service:1.0.2# ... etc. (ucc, ddcs)
Generation rules:
Top-level meta keys (version, generated_at) are not groups — skip.
Every other top-level key is a group. Resolve its NGC source path via the per-group mapping (built-in default or env override).
Within each group:
Keys starting with helm- are OCI helm charts. Skip in the image pass. If INCLUDE_HELM_CHARTS=true, mirror them via OCI helm push (see Helm chart mirroring).
All other keys are container images. The key IS the image name.
The value is either a string tag (scalar) or a list of tags. Mirror each tag separately.
Bash snippet (reference)
#!/bin/bashset -euo pipefail
# Resolve repo root from script location — works regardless of CWD.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"source"$REPO_ROOT/streaming-env.sh"if [[ ! -f "$REPO_ROOT/generated/.env.acr" ]]; thenecho"ERROR: $REPO_ROOT/generated/.env.acr not found. Run the acr skill (stage 04) first." >&2
exit 1
fisource"$REPO_ROOT/generated/.env.acr"
: "${ACR_NAME:?}""${NGC_API_KEY_FILE:?}"
[[ -f "$NGC_API_KEY_FILE" ]] || { echo"ERROR: NGC_API_KEY_FILE='$NGC_API_KEY_FILE' is not a file" >&2; exit 1; }
NGC_SOURCE_REGISTRY="${NGC_SOURCE_REGISTRY:-nvcr.io}"
TARGET_REPOSITORY="${TARGET_REPOSITORY:-$CLUSTER_NAME}"
[[ -n "$TARGET_REPOSITORY" ]] || { echo"ERROR: TARGET_REPOSITORY is empty (no CLUSTER_NAME fallback). Set it in streaming-env.sh." >&2; exit 1; }
MIRROR_TOOL="${MIRROR_TOOL:-regsync}"
INCLUDE_HELM_CHARTS="${INCLUDE_HELM_CHARTS:-false}"
DRY_RUN="${DRY_RUN:-false}"# Manifest keys to skip BY NAME because they are not OCI artifacts and 404 if# mirrored. This is a hand-maintained deny-list, NOT a type lookup: the skill# can't see component_type at runtime — versions.yaml (which carries it) is not# shipped beside manifest.yaml. The default, nvcf-stack, is component_type:# package upstream (an SBOM umbrella label). Durable fix: have the aggregator# emit component_type into manifest.yaml so the type can be read directly.
SKIP_KEYS="${SKIP_KEYS:-nvcf-stack}"
IFS=','read -ra SKIP_KEYS_ARR <<< "$SKIP_KEYS"
MANIFEST_PATH="${MANIFEST_PATH:-$REPO_ROOT/image-manifest/manifest.yaml}"# Temp paths allocated lazily below. Single cleanup trap covers both the# fetched-manifest tempfile and the helm-charts tempdir so concurrent script# runs cannot collide on shared /tmp paths.
FETCHED_MANIFEST=""
CHARTS_DIR=""cleanup() {
[[ -n "$FETCHED_MANIFEST" && -f "$FETCHED_MANIFEST" ]] && rm -f "$FETCHED_MANIFEST"
[[ -n "$CHARTS_DIR" && -d "$CHARTS_DIR" ]] && rm -rf "$CHARTS_DIR"# Always succeed: as the last command in an EXIT trap, a short-circuited &&# guard (e.g. both paths empty in a default run) would otherwise set the# script's exit status to 1 after a fully successful mirror.return 0
}
trap cleanup EXIT
# Per-group NGC source path defaults. Override per-group via env vars named# NGC_SOURCE_REPO_<group> with hyphens replaced by underscores (e.g.# NGC_SOURCE_REPO_nvcf_self_managed_stack). A group with no default and no# override emits a WARN and is skipped — the script still mirrors the rest.# Mirrors image-manifest/source-registry-config.yaml (the source of truth).# Keys are the manifest group names exactly as they appear in manifest.yaml# (e.g. container_cache, not caches).declare -A DEFAULT_SOURCE_REPO=(
["nvcf-self-managed-stack"]="0651155215864979/ncp-dev"
["nvcf-compute-plane-stack"]="0651155215864979/ncp-dev"
["container_cache"]="0651155215864979/ncp-dev"
["gxcache"]="0833294136851237/nvcf-ncp-staging"
["lls"]="0795883928596950"
["ucc"]="nvidia/omniverse"
["ddcs"]="nvidia/omniverse"
["storage-api"]="nvidia/omniverse"
)
resolve_source_repo() {
local group="$1" var="NGC_SOURCE_REPO_${1//-/_}"printf'%s'"${!var:-${DEFAULT_SOURCE_REPO[$group]:-}}"
}
# Resolve manifest — local first, then optionally fetch from GitLab.if [[ ! -f "$MANIFEST_PATH" ]]; thenif [[ "${FETCH_FROM_GITLAB:-false}" == "true" ]]; then
: "${GITLAB_TOKEN:?required when FETCH_FROM_GITLAB=true}"
MANIFEST_REF="${MANIFEST_REF:-main}"echo"--- Fetching manifest from upstream (ref=$MANIFEST_REF) ---"
FETCHED_MANIFEST=$(mktemp -t acr-mirror-manifest.XXXXXX)
curl -fsSL -H "PRIVATE-TOKEN: $GITLAB_TOKEN" \
"https://gitlab-master.nvidia.com/api/v4/projects/omniverse%2Fovonsm%2Fimage-manifest/repository/files/manifest.yaml/raw?ref=${MANIFEST_REF}" \
-o "$FETCHED_MANIFEST"
MANIFEST_PATH="$FETCHED_MANIFEST"elseecho"ERROR: manifest not found at $MANIFEST_PATH" >&2
echo" Either: (a) run from a streaming-staging clone (ships image-manifest/manifest.yaml at repo root)," >&2
echo" (b) set MANIFEST_PATH in streaming-env.sh to a local manifest checkout, or" >&2
echo" (c) set FETCH_FROM_GITLAB=true + GITLAB_TOKEN to fetch from upstream." >&2
exit 1
fificommand -v yq >/dev/null || { echo"ERROR: yq required" >&2; exit 1; }
# The enumeration queries below are written for mikefarah yq v4 (the QA/repo# toolchain). The jq-wrapper yq (python/kislyuk) uses incompatible expression# syntax, so fail clearly rather than emit a confusing lexer error mid-run.
yq --version 2>&1 | grep -qi mikefarah || {
echo"ERROR: this script targets mikefarah yq v4 (https://github.com/mikefarah/yq)." >&2
echo" Found: $(yq --version 2>&1). Install mikefarah yq and re-run." >&2
exit 1
}
echo"--- Manifest source: $MANIFEST_PATH ---"
yq -r '" version: " + .version + " generated_at: " + .generated_at'"$MANIFEST_PATH"# Collect groups (top-level keys minus metadata).mapfile -t ALL_GROUPS < <(yq -r 'keys | .[] | select(. != "version" and . != "generated_at")'"$MANIFEST_PATH")
# Apply INCLUDE_GROUPS filter if set.# NB: do NOT name this array GROUPS — bash auto-populates a special array of# that name with the caller's supplementary group IDs; assignments to it are# dropped, so the loop would iterate GIDs and mirror nothing.if [[ -n "${INCLUDE_GROUPS:-}" ]]; then
IFS=','read -ra REQUESTED <<< "$INCLUDE_GROUPS"
SELECTED_GROUPS=()
for g in"${ALL_GROUPS[@]}"; dofor r in"${REQUESTED[@]}"; do [[ "$g" == "$r" ]] && SELECTED_GROUPS+=("$g") && break; donedoneelse
SELECTED_GROUPS=("${ALL_GROUPS[@]}")
fiecho"--- Groups to process: ${SELECTED_GROUPS[*]} ---"# Auth.echo"--- Authenticating to ACR ---"
az acr login --name "$ACR_NAME"if [[ "$INCLUDE_HELM_CHARTS" == "true" ]]; thenecho"--- Authenticating to NGC for helm OCI ---"
helm registry login "$NGC_SOURCE_REGISTRY" -u '$oauthtoken' --password-stdin < "$NGC_API_KEY_FILE"fi# Build the flat list of (group, image, tag, source-repo) tuples, skipping# helm-* keys (charts handled separately).
TUPLES=() # "<group>\t<image>\t<tag>\t<source-repo>"for group in"${SELECTED_GROUPS[@]}"; do
source_repo=$(resolve_source_repo "$group")
if [[ -z "$source_repo" ]]; then# Hard-fail rather than WARN-skip: a silent skip drops a whole group's# images and only surfaces later as ImagePullBackOff. If a group is# intentionally excluded, use INCLUDE_GROUPS instead.echo"ERROR: no NGC source path for group '$group'. Add it to DEFAULT_SOURCE_REPO or set NGC_SOURCE_REPO_${group//-/_}; use INCLUDE_GROUPS to scope the run." >&2
exit 1
fiwhile IFS= read -r pair; do
image="${pair%%:*}"; tag="${pair#*:}"
skip=""; for s in"${SKIP_KEYS_ARR[@]}"; do [[ "$image" == "$s" ]] && skip=1 && break; doneif [[ -n "$skip" ]]; thenecho"SKIP: $group/$image (non-OCI / package artifact, not mirrorable)" >&2
continuefi
TUPLES+=("${group}\t${image}\t${tag}\t${source_repo}")
done < <(yq -r "
.\"$group\"
| to_entries
| .[]
| select(.key | test(\"^helm-\") | not)
| .key as \$name
| (.value | [.] | flatten | .[]) as \$tag
| \$name + \":\" + \$tag
""$MANIFEST_PATH")
doneecho"--- ${#TUPLES[@]} image:tag pairs to mirror ---"# A silent no-op (0 pairs) is the dangerous failure mode — a later ACR-backed# install then fails with ImagePullBackOff while this step looked successful.# Fail loudly instead, unless the operator deliberately filtered to nothing.if [[ "${#TUPLES[@]}" -eq 0 && -z "${INCLUDE_GROUPS:-}" ]]; thenecho"ERROR: resolved 0 image:tag pairs from $MANIFEST_PATH — nothing to mirror." >&2
echo" Check the manifest has non-helm image keys and that yq enumerated them." >&2
exit 1
fiif [[ "$MIRROR_TOOL" == "regsync" ]]; thenmkdir -p "$REPO_ROOT/generated"
CONFIG="$REPO_ROOT/generated/regsync.yaml"
{
cat <<EOF
version: 1
creds:
# repoAuth: true forces per-repo auth. NGC's namespace returns a fresh
# auth challenge per repo; the cached-token path mishandles this with
# "invalid challenge header".
- registry: ${NGC_SOURCE_REGISTRY}
user: \$oauthtoken
pass: '{{env "NGC_API_KEY"}}'
repoAuth: true
- registry: ${ACR_NAME}.azurecr.io
defaults:
parallel: 4
sync:
EOFfor tuple in"${TUPLES[@]}"; do
IFS=$'\t'read -r group image tag source_repo <<< "$(printf '%b' "$tuple")"cat <<EOF
- source: ${NGC_SOURCE_REGISTRY}/${source_repo}/${image}:${tag}
target: ${ACR_NAME}.azurecr.io/${TARGET_REPOSITORY}/${image}:${tag}
type: image
EOFdone
} > "$CONFIG"echo" wrote $CONFIG"command -v regsync >/dev/null || { echo"ERROR: regsync required (brew install regclient)"; exit 1; }
export NGC_API_KEY
NGC_API_KEY="$(< "$NGC_API_KEY_FILE")"; NGC_API_KEY="${NGC_API_KEY%$'\n'}"if [[ "$DRY_RUN" == "true" ]]; then regsync check -c "$CONFIG"; else regsync once -c "$CONFIG"; fiunset NGC_API_KEY
elif [[ "$MIRROR_TOOL" == "az_acr_import" ]]; then# No extra binary needed; sequential and slower (no parallelism).for tuple in"${TUPLES[@]}"; do
IFS=$'\t'read -r group image tag source_repo <<< "$(printf '%b' "$tuple")"
src="${NGC_SOURCE_REGISTRY}/${source_repo}/${image}:${tag}"
dst="${TARGET_REPOSITORY}/${image}:${tag}"if [[ "$DRY_RUN" == "true" ]]; thenecho" DRY: az acr import --source $src --image $dst (group=$group)"elseecho" [$group] $src → $dst"
az acr import --name "$ACR_NAME" \
--source"$src" --image "$dst" \
--username '$oauthtoken' --password "$(< "$NGC_API_KEY_FILE")" \
--only-show-errors
fidoneelseecho"ERROR: unknown MIRROR_TOOL='$MIRROR_TOOL' (use regsync or az_acr_import)" >&2
exit 1
fi# Helm charts (optional). Same per-group source resolution as for images.if [[ "$INCLUDE_HELM_CHARTS" == "true" ]]; thenecho"--- Mirroring helm charts (OCI) ---"
CHARTS_DIR=$(mktemp -d -t acr-mirror-charts.XXXXXX)
for group in"${SELECTED_GROUPS[@]}"; do
source_repo=$(resolve_source_repo "$group"); [[ -z "$source_repo" ]] && continuewhile IFS= read -r pair; do
chart="${pair%%:*}"; tag="${pair#*:}"if [[ "$DRY_RUN" == "true" ]]; thenecho" DRY: [$group] pull oci://${NGC_SOURCE_REGISTRY}/${source_repo}/${chart}:${tag} → push oci://${ACR_NAME}.azurecr.io/${TARGET_REPOSITORY}"else
helm pull "oci://${NGC_SOURCE_REGISTRY}/${source_repo}/${chart}" --version "$tag" -d "$CHARTS_DIR"
helm push "$CHARTS_DIR/${chart}-${tag}.tgz""oci://${ACR_NAME}.azurecr.io/${TARGET_REPOSITORY}"rm -f "$CHARTS_DIR/${chart}-${tag}.tgz"fidone < <(yq -r "
.\"$group\"
| to_entries
| .[]
| select(.key | test(\"^helm-\"))
| .key as \$name
| (.value | [.] | flatten | .[]) as \$tag
| \$name + \":\" + \$tag
""$MANIFEST_PATH")
donefiecho"--- Done. Verify: az acr repository list --name $ACR_NAME --output table ---"
Helm chart mirroring (optional)
Charts in the manifest are keys prefixed helm- (e.g., helm-nvcf-cassandra: 0.11.1). They are OCI helm artifacts at the source — they need helm pull + helm push, not regsync or az acr import. The script above handles this when INCLUDE_HELM_CHARTS=true.
Why optional: if the cluster has outbound access to nvcr.io and a working NGC auth secret, the helmfile install pulls charts directly from NGC at install time — no mirror needed. Mirror them only for full air-gap where the cluster must not reach nvcr.io.
When mirroring charts, update the helmfile env file:
global:helm:sources:registry:<acr>.azurecr.iorepository:<target-repository># same sub-path as the images
Verification
After mirror, spot-check:
# Image:tag count vs SBOM (across all groups; same exclusion as the mirror — skip helm-*).# Both counts are at the tag level — an image with 2 tags (e.g. nvcf-openbao# during an upgrade transition) contributes 2 to each side.
SBOM_COUNT=$(yq -r '
to_entries | .[]
| select(.key != "version" and .key != "generated_at")
| .value | to_entries | .[]
| select(.key | test("^helm-") | not)
| (.value | [.] | flatten | .[])
| "x"
'"$MANIFEST_PATH" | wc -l | tr -d ' ')
ACR_COUNT=0
while IFS= read -r repo; do
[[ -z "$repo" ]] && continue
tag_count=$(az acr repository show-tags --name "$ACR_NAME" --repository "$repo" -o tsv 2>/dev/null | wc -l | tr -d ' ')
ACR_COUNT=$((ACR_COUNT + tag_count))
done < <(az acr repository list --name "$ACR_NAME" -o tsv | grep "^${TARGET_REPOSITORY}/")
echo"SBOM expects: $SBOM_COUNT image:tag pair(s) / ACR has: $ACR_COUNT under ${TARGET_REPOSITORY}/"# Spot-check a tag from any group
az acr repository show-tags --name "$ACR_NAME" --repository "${TARGET_REPOSITORY}/bitnami-cassandra" -o tsv
If counts diverge, re-run with DRY_RUN=true to inspect the planned sync list (each entry is prefixed with its [group]), then re-run for real.
Teardown
Mirror state is just data in ACR. Either:
az acr repository delete --name "$ACR_NAME" --repository "<sub-path>/<image>" --yes per image (when retiring specific images), or
Let the ACR itself be torn down by the acr skill's --teardown (which destroys the registry and everything in it).
There is no acr-mirror --teardown because the deliberate destructive action lives in the foundation skill.
Validation checklist
Script opens with set -euo pipefail and resolves repo root via BASH_SOURCE
Sources streaming-env.sh and .env.acr (guard: error if stage 04 hasn't run)
Required vars (ACR_NAME, NGC_API_KEY_FILE) have fail-fast checks; no other required inputs
NGC_API_KEY_FILE is verified to be a file ([[ -f ]]) before any redirection — the leak-postmortem pattern from earlier in this project
DEFAULT_SOURCE_REPO table is present and covers every group currently in manifest.yaml (nvcf-self-managed-stack, nvcf-compute-plane-stack, container_cache, gxcache, lls, ucc, ddcs, storage-api) and matches source-registry-config.yaml
An unmapped non-metadata group is a hard error (exit 1), not a WARN-skip
cleanup() ends with return 0 so the EXIT trap can't fail a successful run
resolve_source_repo honors per-group env override (NGC_SOURCE_REPO_<GROUP> with hyphens→underscores) before falling back to the default table
Unknown groups emit WARN and are skipped — the script does NOT fail on them
Manifest path resolves with the streaming-staging-clone default; fallback to GitLab fetch is gated behind FETCH_FROM_GITLAB=true
Tempfiles for the fetched manifest and helm-charts directory are allocated via mktemp / mktemp -d (never fixed /tmp/<name> paths) and a single trap cleanup EXIT removes them on every exit path — safe under concurrent runs
Manifest provenance is echoed (version, generated_at) for audit
Top-level metadata keys (version, generated_at) are NOT treated as groups
INCLUDE_GROUPS filter (comma-separated) subsets the groups when set
SKIP_KEYS (default nvcf-stack) drops non-OCI package-type keys from the image pass with a logged note
helm-* keys are skipped in the image pass
Both scalar and list tag values handled (verify against the nvcf-openbao upgrade-transition case)
TARGET_REPOSITORY is non-empty (defaults to ${CLUSTER_NAME}); explicit fail-fast guards against empty (e.g., if both env var and CLUSTER_NAME are unset)
"Why the sub-path matters" covers both the double-slash and the multi-deployment collision risk
MIRROR_TOOL=regsync emits regsync.yaml with repoAuth: true on the NGC creds entry (NGC's per-repo auth challenge breaks token caching)
MIRROR_TOOL=az_acr_import works without regsync installed
Each generated mirror entry (regsync or az acr import) uses the per-group source repo, not a single hardcoded one
DRY_RUN=true produces output without pushing anything; output is prefixed with [group] so per-group entries are visible
INCLUDE_HELM_CHARTS=true mirrors OCI helm artifacts via helm pull + helm push, with the same per-group source resolution
Verification snippet (image-count compare across all groups) is documented
