| name | vp-deploy-test |
| description | AI assistance for validating a Validated Pattern deployment end-to-end — verifying VP Operator installation, ArgoCD Application convergence, secrets delivery via Vault and ESO, and imperative job completion. Use after pattern.sh make install or VP Operator install, before running student-readiness checks. |
| related_skills | ["patternizer","vp-refactor","student-readiness","workshop-tester","vp-deploy-validator","vp-submission"] |
Validated Pattern Deployment Tester Skill
When to Use
Purpose: Use this skill to build confidence that the Validated Pattern deployment is sound — that installation succeeded, all Applications converged, secrets are flowing, and jobs completed — before handing the environment to students or running workshop-tester. Passing all phases means an operator can run this pattern with limited or no issues.
- After
pattern.sh make install or VP Operator installation completes and you want to verify the pattern is fully deployed
- ArgoCD Applications were created but you're not sure all are Healthy/Synced
- Secrets are not reaching application pods (ESO or Vault issues)
- Imperative jobs in the
jobs/ directory have not run or are failing
- Before running student-readiness or handing the environment to students
- After fixing a
vp-refactor audit finding and re-deploying
Do NOT use this skill to initialize a new pattern from scratch — use the patternizer skill instead. Do NOT use this skill to audit pattern files without a live deployment — use the vp-refactor skill instead. To check an already-running deployment without re-installing, use the vp-deploy-validator skill instead.
Instructions
This skill defines a four-phase process. Work through the phases in order. Do not proceed to the next phase if the current phase has unresolved failures.
Required Input
Before starting, collect:
| Input | Required | Example |
|---|
| Pattern repository path or URL | Yes | ~/my-pattern/ or GitHub URL |
| Cluster group name | Yes | hub, datacenter |
| OpenShift cluster access | Yes | oc login token or kubeconfig path |
| Installation method | Yes | operator or cli (pattern.sh) |
| Operators the pattern installs | Yes | ACM, OpenShift GitOps, cert-manager |
| Whether the pattern has a workshop | No — defaults to no | yes, no |
Phase 1 — Pre-flight
Verify the cluster and pattern repo are ready before installing.
| Check | Command | Pass condition |
|---|
| OCP login | oc whoami | Returns a username |
| Cluster version | oc version | 4.12 or higher (VP minimum) |
| OpenShift GitOps installed | oc get csv -n openshift-operators | grep gitops | CSV in Succeeded state |
| VP Operator installed | oc get csv -n openshift-operators | grep patterns | CSV in Succeeded state (operator install method) |
| values-global.yaml present | ls values-global.yaml | File exists |
| values-.yaml present | ls values-<cluster-group>.yaml | File exists |
| values-secret.yaml NOT in git | git status values-secret.yaml | Not tracked / in .gitignore |
| values-secret.yaml present locally | ls values-secret.yaml | File exists (outside git) |
If values-secret.yaml is tracked by git, stop immediately:
CRITICAL: values-secret.yaml is committed to git.
This file contains secrets and must never be committed.
→ Remove it from git history before proceeding:
git rm --cached values-secret.yaml
echo "values-secret.yaml" >> .gitignore
git commit -m "remove secrets from git"
If any other pre-flight check fails, stop and provide the corrective action. Do not proceed to Phase 2.
Phase 2 — Install
Run the pattern installation and capture the result.
CLI install:
Before running pattern.sh make install directly, check whether a deploy.sh script already exists in the pattern repo root:
ls deploy.sh
If deploy.sh exists: run it instead — it includes the values-secret.yaml pre-check required for non-interactive install validation:
./deploy.sh
If deploy.sh does not exist: run pattern.sh make install directly, then recommend generating the script using the patternizer skill's "Independent Deployment Scripts" section:
cd ~/my-pattern
./pattern.sh make install
VP Operator install:
Confirm the operator has picked up the GitOps repo:
oc get pattern -n openshift-operators
Expected: a Pattern CR exists and shows the target GitOps repo URL.
Capture:
- Exit code for CLI install (0 = success, non-zero = failure)
- Whether the ArgoCD ApplicationSet was created
- Whether any interactive prompt appeared in the output
oc get applicationset -n openshift-gitops
Non-interactive install check (CLI method only):
While streaming or reviewing the pattern.sh output, watch for any of these patterns indicating user interaction was required:
- A line containing
[y/N], [Y/n], (yes/no), Enter password:, Password:, or Select with a numbered choice
- The process pausing without output for more than 30 seconds (TTY wait)
- A
read prompt from a shell script
- An
oc login prompt requesting credentials
If any interactive prompt is detected:
Deployment Test — Phase 2: SUBMISSION_BLOCKING — Non-interactive install failure
Pattern: <pattern-name>
Prompt: <exact prompt text>
A Validated Pattern must deploy fully unattended via pattern.sh or the VP
Operator to be accepted at any VP tier. This is a submission-blocking issue.
Action: Locate and remove all interactive prompts from:
- pattern.sh and all Makefiles it calls
- Any shell scripts called during install
- values files (no prompts for missing values — all must have defaults
or be provided via values-secret.yaml)
Common causes:
- `read` calls in install helpers without a default
- oc login without pre-configured credentials
- Helm `--set` prompts for required but unset values
- `confirm.sh` style scripts
After fixing: re-run this phase to verify a clean unattended install.
Record this as SUBMISSION_BLOCKING and stop. Do not proceed to Phase 3 until the install is confirmed non-interactive.
If install fails (non-zero exit, no interactive prompt):
Deployment Test — Phase 2: Install FAILED
Pattern: <pattern-name>
Method: <cli|operator>
Exit code: <N>
Error: <last error line>
Next step: activate vp-refactor to audit the pattern structure.
Common causes:
- main.clusterGroupName in values-global.yaml does not match a values-<name>.yaml filename
- values-<cluster>.yaml missing clusterGroup.applications or namespaces blocks
- pattern-metadata.yaml missing or malformed
- OpenShift GitOps not installed before pattern.sh make install
Stop and report. Do not proceed until the user has resolved the failure and re-installed.
Phase 3 — ArgoCD Convergence
After the ApplicationSet is created, poll until all ArgoCD Applications reach Healthy/Synced. VP deployments are async — convergence may take several minutes.
oc get applications -n openshift-gitops
oc get applications -n openshift-gitops \
-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.health.status}{"\t"}{.status.sync.status}{"\n"}{end}'
Poll every 30 seconds, up to 10 minutes. Report status at each interval.
Convergence success criteria:
- All Applications show
health: Healthy and sync: Synced
If convergence fails (timeout or Degraded/OutOfSync):
For each failing Application, collect:
oc describe application <app-name> -n openshift-gitops
Classify the failure:
| Symptom | Likely cause | Action |
|---|
SyncFailed: ComparedTo error | Helm values rendering error | Check values file structure → escalate to vp-refactor |
OutOfSync: Unknown | ArgoCD can't reach Git repo | Check repo credentials in ArgoCD → verify GitOps repo URL |
Degraded: Missing resource | Chart missing a required CRD or operator | Check clusterGroup.subscriptions in values file |
Degraded: OperatorNotInstalled | OLM subscription not resolving | Verify catalog source and package name → escalate to vp-refactor |
Progressing (stuck) | Resource creation timing issue | Check sync wave ordering in charts |
Deployment Test — Phase 3: ArgoCD Convergence FAILED
Application: <app-name>
Health: Degraded
Sync: OutOfSync
Reason: <ArgoCD error message>
Next step: activate vp-refactor to audit values files and chart structure.
Phase 3 Convergence Report
Deployment Test — Phase 3: ArgoCD Convergence
──────────────────────────────────────────────────────
Application Health Sync Notes
<pattern>-hub Healthy Synced —
<pattern>-acm Healthy Synced —
<pattern>-app1 Degraded OutOfSync Helm render error: missing ingress.host
──────────────────────────────────────────────────────
Converged: <N>/<N> Applications
Phase 4 — Secrets and Jobs Verification
After convergence, verify that secrets are flowing and imperative jobs have run.
4a. Secrets delivery (Vault + ESO)
Check ExternalSecret resources are in Ready state:
oc get externalsecrets -A
Expected: all ExternalSecrets show READY=True and STATUS=SecretSynced.
If any ExternalSecret shows SecretSyncedError:
oc describe externalsecret <name> -n <namespace>
Common causes:
- Vault is not initialized or unsealed
- ESO SecretStore credentials are incorrect
- The secret path in Vault does not match the ExternalSecret spec
4b. Imperative jobs
If the pattern has a jobs/ directory, check that CronJobs have completed at least one successful run:
oc get cronjobs -n <imperative-namespace>
oc get jobs -n <imperative-namespace>
Expected: at least one Job per CronJob with COMPLETIONS: 1/1.
If jobs are not running or failing:
- Verify the CronJob schedule is not
never (patterns sometimes use @once for bootstrap jobs)
- Check job logs:
oc logs job/<job-name> -n <namespace>
4c. Student-readiness hand-off
After secrets and jobs checks pass, activate the student-readiness skill:
Phase 4 checks passed. Activating student-readiness for environment validation.
→ Use the student-readiness skill with cluster: <api-url>
If student-readiness fails:
student-readiness reported failures. Likely causes:
- An Application converged but its workload pods are CrashLooping
- A secret was not delivered to the expected namespace
→ Escalate to vp-refactor to audit the values files and chart structure
Phase 4 Report
Deployment Test — Phase 4: Secrets and Jobs
──────────────────────────────────────────────────────
Check Status Notes
ExternalSecret: <name> PASS SecretSynced
ExternalSecret: <name> FAIL SecretSyncedError: Vault path not found
CronJob: <name> PASS 1/1 completed
student-readiness PASS —
──────────────────────────────────────────────────────
After all Phase 4 checks pass: the deployment is confirmed sound. The next step in the confidence chain is workshop-tester:
All deployment tests passed.
→ Activate workshop-tester to validate module exercises against this environment.
This is the next step in building operator confidence before going live.
Final Summary Report
Validated Pattern Deployment Test — Complete
══════════════════════════════════════════════════════
Pattern: <pattern-name>
Cluster: <api-url>
Method: <cli|operator>
Phase 1 — Pre-flight PASS
Phase 2 — Install PASS
Phase 3 — ArgoCD Convergence PASS / FAIL
Applications converged <N>/<N>
Phase 4 — Secrets and Jobs PASS / FAIL
ExternalSecrets <N>/<N> Ready
CronJobs <N>/<N> completed
student-readiness PASS / FAIL
══════════════════════════════════════════════════════
Overall: READY FOR STUDENTS / NEEDS ATTENTION
Confidence: HIGH — all phases passed. An operator can run this pattern
with limited or no issues.
Next step: run workshop-tester to validate module exercises.
Remediation Plan
When the overall result is NEEDS ATTENTION, generate an ordered, prioritized remediation plan from the actual failures found. Create each item as a session todo so progress is tracked.
Prioritize in this order:
| Priority | Condition | Action |
|---|
| BLOCKING | values-secret.yaml committed to git (Phase 1) | Remove from git history immediately — do not proceed until resolved |
| BLOCKING | Install failed / ApplicationSet not created (Phase 2) | Activate vp-refactor to audit values files, chart structure, and pattern-metadata.yaml |
| BLOCKING | ArgoCD Application Degraded/OutOfSync — SyncFailed (Phase 3) | Activate vp-refactor, Audit Area 1 (Values) and Area 3 (Charts) for the specific failing Application |
| HIGH | ArgoCD Application OutOfSync — Git repo unreachable (Phase 3) | Verify ArgoCD repo credentials and GitOps repo URL in the pattern CR |
| HIGH | ExternalSecret SecretSyncedError (Phase 4) | Activate vp-refactor, Audit Area 4 (Secrets Management); check Vault path and ESO SecretStore credentials |
| HIGH | student-readiness failed (Phase 4) | Work through student-readiness troubleshooting tree; escalate to vp-refactor if root cause is in the pattern |
| MEDIUM | Imperative CronJob not completing (Phase 4) | Activate vp-refactor, Audit Area 8 (Imperative Jobs); check job logs |
| LOW | ArgoCD Application stuck Progressing (Phase 3) | Check sync wave ordering in charts; may resolve with a manual ArgoCD sync retry |
Present the plan as an ordered list containing only the items that actually failed:
Deployment Remediation Plan — <pattern-name>
──────────────────────────────────────────────────────
Priority # Finding Action
BLOCKING 1 hub app SyncFailed (Helm render error) Activate vp-refactor → Audit Area 3
HIGH 2 ExternalSecret db-creds SecretSyncedError Activate vp-refactor → Audit Area 4
MEDIUM 3 CronJob bootstrap-job not completing Check job logs, vp-refactor Audit Area 8
──────────────────────────────────────────────────────
Created 3 todos. Resolve in order — BLOCKING items first.
After presenting the plan, ask:
"Would you like to start with item 1 now? (y/n)"
If yes, activate the indicated skill immediately.
Re-test After Fixes
After the developer resolves one or more items from the remediation plan, re-run only the phases that contained failures rather than a full reinstall:
Re-running failed phases only:
Phase 3 (ArgoCD convergence) — re-checking Application health after Helm fix
Phase 4 (Secrets and jobs) — re-checking ExternalSecrets and CronJobs
Skipping Phase 1 (pre-flight passed) and Phase 2 (install succeeded).
For ArgoCD re-checks, trigger a manual sync first rather than re-installing:
argocd app sync <app-name> --server <argocd-url>
oc patch application <app-name> -n openshift-gitops \
--type merge -p '{"operation":{"sync":{}}}'
Compare results against the previous run and show which items were resolved:
Re-test Results — <pattern-name>
──────────────────────────────────────────────────────
Item Finding Previous Now
1 hub app SyncFailed FAIL → PASS ✓ resolved
2 db-creds SecretSyncedError FAIL → FAIL still failing
3 bootstrap-job not completing FAIL → PASS ✓ resolved
──────────────────────────────────────────────────────
Resolved: 2/3 Remaining: 1
Repeat until all items resolve, then present the full Final Summary Report as READY FOR STUDENTS.
Escalation
- Install failure or ApplicationSet not created → Use the vp-refactor skill to audit values files, chart structure, and pattern-metadata.yaml
- ArgoCD Application Degraded/OutOfSync → Use the vp-refactor skill, Audit Area 3 (Charts Directory) and Audit Area 1 (Values File Completeness)
- ExternalSecret errors (Vault/ESO) → Use the vp-refactor skill, Audit Area 4 (Secrets Management)
- Imperative jobs failing → Use the vp-refactor skill, Audit Area 8 (Imperative Jobs)
- student-readiness failures after successful convergence → Use the student-readiness skill's troubleshooting decision tree; if the root cause is in the pattern, escalate to vp-refactor
