rebundle

name	rebundle
description	Reset a previously-configured bundle to raw state so CI can re-run the full pipeline. Restores originals, removes pipeline-generated files, and pushes for CI. Use when a bundle needs to be re-processed through automation after pipeline or validator changes.
user-invocable	true

Rebundle

Resets a previously-configured bundle to its raw asset state so CI can re-run the full pipeline from scratch. This is needed when pipeline scripts, validator logic, or raw assets have changed since the bundle was last processed.

When to use

• Pipeline scripts (scripts/configurator/, scripts/run_pipeline.py) have been updated on main
• Validator logic (src/validate/) has changed
• Raw assets need corrections (e.g., missing transform columns, summary SQL fixes)
• A bundle was configured by an older version of the pipeline and needs regeneration

Step 1: Identify the bundle

If the user didn't specify a bundle directory, ask which bundle to rebundle using AskUserQuestion:

• Scan aws/ and trafficpeak/ for directories containing bundle-config.json
• Present the options

Store the bundle directory path (e.g., trafficpeak/bot_insights_siem).

Step 2: Ensure branch is up to date with main

This is critical — the whole point of rebundling is often to pick up pipeline changes from main.

git fetch origin
git merge origin/main --no-edit

If there are merge conflicts, stop and ask the user to resolve them.

Step 3: Analyze git history

Run these in parallel:

• git log --oneline HEAD -- {bundle_dir}/ — full commit history for this bundle
• git log --oneline HEAD -- .originals/{bundle_dir}/ — originals history
• find {bundle_dir}/ -type f | sort — current bundle files
• find .originals/{bundle_dir}/ -type f 2>/dev/null | sort — current originals files

Classify each commit as either:

• Raw/manual — commits by a human (contributor assets, manual fixes)
• Pipeline-generated — commits by github-actions[bot] with [skip ci] prefix

Identify:

1. Which files are raw assets (dashboard JSONs, summary SQLs, transform JSONs from contributors)
2. Which files are pipeline-generated (bundle.json, sample_data.json, renamed transform.json)
3. Whether .originals/ exists and what state it's in

Step 4: Fix .originals/ structure if needed

Compare the .originals/ directory structure to the current bundle directory structure:

• If .originals/ has a version subdirectory (e.g., 1.0.0/) but the bundle is flat, flatten .originals/
• If .originals/ doesn't exist at all, this bundle may never have been through the pipeline — warn the user and ask how to proceed

The .originals/ path must mirror the bundle path exactly (this is how originals_manager.py computes paths via os.path.relpath).

Step 5: Check for raw asset changes needed

Compare raw assets in .originals/ against the pipeline-modified versions in the bundle directory. Look for:

• Columns added to transforms that aren't in .originals/ (e.g., computed columns added post-pipeline)
• Summary SQL changes that reflect schema updates
• Dashboard structural changes made manually after pipeline ran

If the user mentioned specific changes or provided documentation about required asset modifications, apply those changes to the .originals/ files first. The originals must contain the corrected raw assets before restoring.

Important: Always confirm with the user before modifying .originals/ — these are the source of truth for raw assets.

Step 6: Restore raw state

Once .originals/ is correct, reset the bundle working directory. This must mirror what restore_from_originals() in scripts/originals_manager.py does — a full wipe, not selective deletion.

1. Delete everything in the bundle directory except bundle-config.json:

   find {bundle_dir}/ -type f ! -name 'bundle-config.json' -delete
   find {bundle_dir}/ -type d -empty -delete
   

   This removes ALL pipeline artifacts and configured files — not just the obvious ones like bundle.json and sample_data.json. Dashboard files with injected __DATASOURCE__ markers, summaries with template variables, and any other modified files are all removed.

2. Copy everything from .originals/ into the bundle directory:

   cp -r .originals/{bundle_dir}/* {bundle_dir}/
   

   This restores raw dashboards, summaries, and transforms with original filenames and content.

3. Verify the bundle directory contains only:

   • bundle-config.json (kept — triggers Track 1 in CI)
   • Raw dashboard(s) in dashboards/
   • Raw summary SQL(s) in summaries/ (if any)
   • Raw transform(s) in transformations/ (original filenames)

4. Verify raw state by spot-checking:

   • Summary SQL should have raw table references (e.g., akamai.siem), NOT template variables
   • Transform should have original filename, NOT transform.json

Step 6b: Verify track detection

Confirm CI will route this bundle to the full pipeline, not validate-only:

python3 scripts/detect_track.py --bundle-dir {bundle_dir} --changed-files {bundle_dir}/bundle-config.json --mode track

The output must be full. If it returns validate-only or ambiguous, the restore is incomplete — residual configured markers remain. Stop and diagnose before proceeding.

Step 7: Stage selectively

Do NOT use git add -A — there may be unrelated untracked files (.claude/worktrees/, local portables artifacts, etc.).

Stage only:

• .originals/{bundle_dir}/ changes (flattening, raw asset updates)
• {bundle_dir}/ changes (raw restore, pipeline file removal)

Step 8: Ship

Invoke the /ship skill to commit, push, open/update the PR, and notify Slack. The /ship skill will handle the commit message prompt, code review, PR creation, and Slack notification.